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(57) Abstract 

aav.Tnlor*2 '"'f *^ protection of streamed media content is disclosed. TTic apparatus includes control means for 

LTSk m^ffL'^^n ^.'^^"^ objec^under control of the contn>! means. 

^ n!^.^ ^ ^ ^ acmal use of content streams or objects. Hie control means may operate in accordance witfi rules received 
f It ! ^ "^""^^ °^ ? side^band channel TTie rules may specify allowed uses of the content, inchiding whether or 

not the content be copied or transferred, and whether and under what circumstances received content may be "checked out" of on^ 
"^T*^ '"^^^ °^ ^^SCls. and a requirement that au<iit information be collected 

and/or tensmitted to an external server. TTie apparatus may include a media player designed to call plugins to assist in renden^ig co«^ 
wi^/^T ^ ^ ^ ^i^>ofl so that a media pteyer designed for use with unprotected conlent may render^Sl^iS^; 
?S:^N^^&1^^^^ l^streamed content may be inanumber of Afferent fZats. includ^g 



FOR THE PURPOSES OF INFORMATION ONLY 



Codes used to identify States party to the PCT on the front pages of pamphlets publishing international applications under the PCT. 



AL 


Albania 


ES 


Spain 


LS 


AM 


Armeaia 


n 


Finland 


LT 


AT 


Austria 


FR 


France 


LU 


AU 


Australia 


GA 


Gabon 


LV 


AZ 


Azeibatjan 


GB 


United Kingdom 


MC 


BA 


Bosnia and Herzegovina 


GE 


Georgia 


MD 


BB 


Bafbados 


GH 


Ghana 


MG 


BE 


Belgium 


GN 


Guinea 


MK 


BF 


BuHctna Faso 


GR 


Greece 


BG 


Bulgaria 


HU 


Hungary 


ML 


BJ 


Benin 


IE 


Ireland 


MN 


BR 


Bmil 


IL 


Israel 


MR 


BY 


Be lams 


IS 


Iceland 


MW 


CA 


Canada 


IT 


Italy 


MX 


CF 


Central African Republic 


JP 




NB 


CC 


Congo 


KE 


Kenya 


NL 


CH 


Switzerland 


KG 


Kyigyzstan 


NO 


a 


Cftted'lvDtic 


KP 


Democratic Pe(q>le*i 


NZ 


CM 


CanterooQ 




Republic of Korea 


PL 


CN 


China 


KR 


R^bitc of Kocea 


FT 


cu 


Cuba 


KZ 




RO 


cz 


Czech Republic 


LC 


Saint Lucia 


RU 


DE 


Germany 


U 


Uedttenstehi 


SD 


OK 


Denmark 


LK 


Sri Lanka 


S£ 


EE 


Estonia 


LR 


Liberia 


SG 



Lesotho 


SI 


Stovenia 


\ ithmt^xn 


SK 


Stovakia 


Luxembourg 


SN 


Senegal 


Latvia 


sz 


Swaziland 


Monaco 


TD 


Chad 


Republic of Moldova 


TG 


Togo 


Madagascar 


TJ 


Tajikistan 


Tlie former Yugoslav 


TM 


Ttukmenistan 


Republic of Macedonia 


TR 


Tmkgy 


Mali 


TT 


Trinidad and Tobago 


Mongolia 


UA 


Ukraine 


Mauritania 


UG 


Uganda 


Malawi 


US 


United States of America 


Mexico 


uz 


Urf>ftkistan 


Niger 


VN 


Viet Nam 


Nethertaiub 


YU 


Yogoshvia 


Norway 


ZW 


Zimbabwe 



New Zealand 
Poland 
Portugal 
Romania 

Russian Federation 
Sudan 
Sweden 
Singapore 



wo 99/48296 



PCTAJS99/0S734 



-1- 



METHODS AND APPARATUS FOR CONTINUOUS CONTROL AND PROTECTION OF MEDIA CONTENT 

FIELD OF THE TNVFNTmiv 

TWs invention relates generally to computer and/or electronic security. More 
particularly, this invention relates to systems and methods for protection of information in 
streamed format. 
BACKGROUND 

Streaming digital media consists generally of sequences of digital information 
received in a "stream" of packets, and designed to be displayed or rendered Examples 
include streamed audio content, streamed video, etc. 

Digital media streams are becoming an increasingly significant means of content 
delivery, and foma the basis for several adopted, proposed or de facto standards. TTie 
acceptance of this format, however, has been retarded by the ease with which digital media 
streams can be copied and improperly disseminated, and the consequent reluctance of 
content owners to allow significant properties to be distributed through streaming digital 
means. For this reason. Aere is a need for a methodology by which digital media streams 
can be protected. 

SUMMARY OF THE TNVENTmN 

Consistent with the invention, this specification describes a new architecture for 
protection of information provided in streamed format. This architecture is described in the 
context of a generic system which resembles a system to render content encoded pursuant 
to the MPEG-4 specification (ISO/IEC 1 4496.1). though with certain modifications, and 
with the proviso fliat the described system may differ from the MPEG-4 standard in certain 
respects. A variety of different embodiments is described, including an MPEG-4 
embodiment and a system designed to render content encoded pursuant to the MP3 
specification (ISO/IEC TR 1 1 172). 

According to aspects of the invention, this architecture involves system design 
aspects and information format aspects. System design aspects include the incorporation of 
content protection fimctionality. control fimctionaUty. and feedback enabling control 
fimctionality to monitor the activities of the system. Information fonnat aspects include 
the incorporation of rale/control infommion into information streams, and the protection of 
content through mechanisms such as encryption and watermaridng. 
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Systems and methods consistent with the present invention perfoim content 
protection and digital rights management. A streaming media player consistent with the 
present mvention includes a port designed to accept a digital bit stream. n.e digital bit 
stream mcludes content, which is encrypted at least in part, and a secure container 
mcludmg control information designed to control use of the content, including at least one 
key suitable for decryption of at least a portion of the content. The media player also 
mcludes a control arrangement including a means for opening secure containers and 
extractmg ayptographic keys, and means for decrypting the encrypted portion of the 
content. 

BRIEF DFSrPTPTinM pp trf. HP awim/ic 

The accompanying drawings, which are incorporated in and constitute a part of this 
spec.ficat.on. iUustrate an embodiment of the invention and. together with the description 
serve to explain the advantages and principles of the invention. In the drawings, 

FIG. 1 shows a generic system consistent with the present invention; 

FIG. 2 shows an exemplary Header 201 consistent with the present invention; 

FIG. 3 shows a general encoding format consistent widi the present invention; 

FIG. 4 iUustrates one mamter for storing a representation of a work consistent'with 
the presoit invention; 

FIG. 5 shows an example of a control message format; 

FIG. 6 is a flow diagram illustrating one embodiment of the steps which take place 
using the functional blocks of FIG. 1; 

FIG. 7 illustrates a form wherein the control messages may be stored in Control 
Block 13; 

FIG. 8 shows MPEG-4 System 801 consistent with the present invention; 
FIG. 9 shows an example of a message format; 
FIG. 10 iUustrates an PMP table consistent with the present invention; 
FIG. 1 1 iUustrates a system consistent with die present invention; 
FIG. 12 iUustrates one embodiment of the DigiBox format; 
FIG. 13 shows an example of a Real Networks file format (RMFF); 
FIG. 14 shows an RNPFF format consistent with the present invention; 
FIG. 15 illustrates the flow of changes to data in die Real Networks fill format in 
an architecture consistent with the present invention; 

FIG. 16 illustrates a standard Real Networics architecture; 

na 1 7 shows an exemplary architecture in which a trust plugin operates within the 
overaU Real Networks architecture; 
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FIG. 18 shows a bit stream format consistent with the principles of the present 
invention; 

FIG. 19 shows one embodiment of protection applied to the MP3 format; 
FIG. 20 illustrates one embodiment of an MPS player designed to process and 
render protected content; 

FIG. 2 1 illustrates the flow of data in one embodiment in which a protected MPEG- 
4 file may be created consistent with the present invnetion; 

FIG. 22 iUustrates the flow of data in one embodiment in which control may be 
incorporated into an existing MPEG-4 stream consistent with the present invention; 

FIG. 23 shows a system consistent with the principles of the present invention; 

FIG. 24 shows a system consistent with the principles of the present invention;' 

FIG. 25 shows an example of an aggregate stream consistent with the present ' 
invention; 

FIG. 26 illustrates a Header CMPO 2601 consistent with the present invention; 
FIG. 27 shows exemplary Content Management Protection Objects consistent with 
the principles of the present invention; and 

FIG. 28 shows an example of a CMPO Data Structure 2801 consistent with the 
present invention. 

DETAn.F.D DESCRTPTTniNJ 
Reference will now be made in detail to implementations consistent with the 
principles of the present invention as illustrated in the accompanying drawings. 

The following U.S. patents and applications, each of which is assigned to the 
assignee of the current application, are hereby incorporated in their entirety by reference: 
Ginter. et al.. "Systems and Methods for Secure Transaction Management and Electronic 
Rights Protection." U.S. Patent AppUcation Serial No. 08/964,333, filed on November 4 
1997 ("Ginter '333"); Ginter. et al., "Trusted Infrastructure Support Systems. Meftods arid 
Techniques for Secure electronic commerce. Electronic Transactions, Commerce Process 
Control Automation, Distributed Computing, and Rights Management. " U.S. Patent 
Application Serial No. 08/699.712. filed on August 12. 1996 ("Ginter '712") ; Van Wie. et 
al, "Steganographic Techniques for Securely Delivering Electronic Digital Rights 
Management Information Over Insecure Communications Channels. U.S. Patent 
Application Serial No. 08/689,606, filed on August 12. 1996 ("Van Wie") ; Ginter, et. al 
"Software Tamper Resistance and Secure Communication." U.S. Patent Application Serial 
No. 08/706.206. filed on August 30. 1996 ("Ginter, '206"); Shear, et al. "Cryptographic 
Methods. Apparatus & Systems for Storage Media Electronic Rights Management in 
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15 



Closed & Connected Appliances, " U.S. Patent Application Serial No. 08/848,077. filed on 
May 15, 1997 ("Shear"); Collberg et al. "Obfuscation Techniques for Enhancing Software 
Security, " U.S. Patent Application Serial No. 09/095,346, filed on June 9, 1998 
("Collberg"); Shear. "Database Usage Metering and Protection System and Method," U.S. 
Patent No. 4,827,508. issued on May 2, 1989 ("Shear Patent"). 

HG. I illustrates Media System 1, which is capable of accepting, decoding, and 
rendering streamed multimedia content. This is a generic system, though it includes 
elements based on the MPEG-4 specification. Media System 1 may include software 
modules, hardware (including integrated circuits) or a combination. In one embodiment, 
Media System 1 may include a Protected Processing Environment (PPE) as described in 
the Ginter '333 application. 

In FIG. 1, Bit Stream 2 represents input information received by System I. Bit 
Stream 2 may be received through a connection to an external network (e.g., an Internet 
connection, a cable hookup, radio transmission &om a satellite broadcaster, etc.), or may be 
received fix)m a portable memory device, such as a DVD player. 

Bit Stream 2 is made up of a group of related streams of information, including 
Organization Stream 3, Audio Stream 4, Video Stream 5, Control Stream 6, and Info 
Stream 31. Each of these streams is encoded into the overaU Bit Stream 2. Each of these 
represents acategory of streams, so that, for example, Video Stream 5 may be made up of a 
20 number of separate Video Streams. 

These streams correspond generaUy to streams described in the MPEG-4 format as 
follows: 

Organization Stream 3 corresponds generally to the BIFS stream and the OD 
("Object Descriptor") stream. 

Audio Stream 4 and Video Stream 5 correspond generally to the Audio and Video 
streams. 

Control Stream 6 corresponds gaierally to the IPMP stream. 
Audio Stream 4 includes compressed (and possibly encrypted) digital audio 
information. This information is used to create the sound rendered and output by Media 
System 1. Audio Stream 1 may represent multiple audio streams. These multiple streams 
may act together to make up the audio output, or may represent alternative audio outputs. 

Video Stream 5 includes compressed (and possibly encrypted) digital video 
information. This information is used to create the images and video rendered and output 
by Media System 1. Video Stream 5 may represent multiple video streams. TTiese 
multiple streams may act together to make up the video output, or may represent alternative 
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video outputs. 

Organization Stream 3 includes organizational information and metadata related to 
the work to be rendered. TTus infomiation may include a tree or other organizational 
device which groups audio and video streams into objects. This infonnation may also 
include metadata associated with the entire work, the objects, or the individual streams. 

Control Stream 6 includes control infomiation. divided generally into header 
infonnation and messages. The header infomiation includes an identifier for each discrete 
message. The content of the messages, which will be described further below, may include 
cryptographic keys and rales goveming the use of content. 

Info Stream 3 1 carries additional information associated with the content in other 
components of Bit Stream 2, including but not limited to graphics repr^enting cover art. 
text for lyrics, coded sheet music or other notation, independent advertising content, 
concert information, fan club information, and so forth. Info Stream 31 can also ca^ 
system management and control information and/or components, such as updates to 
software or fimiware in Media System 1. algorithm implementations for content-specific 
functions such as watermarking, etc. 

Each of these streams is made up of packets of infonnation. In one exemplary 
embodiment, each packet is 32 bytes in length. Since a single commmiications chamiel 
(e.g., a cable, a bus. an infiared or radio comiection) contains packets from each of the 
streams, packets need to be identified as belonging to a particular stream. In a preferred 
embodiment, this is done by including a header which identifies a particular stream and 
specifies the number of following packets which are part of that stream. In another 
embodiment, each packet may include individual stream infonnation. 

Exemplary Header 201 is shown in FIG. 2. This header may generally be used for 
the Organization, Audio and Video Streams. A header for the Control Stream is described 
below. Header 201 includes Field 202, which includes a bit pattern identifying Header 201 
as a header. Field 203 identifies the particular type of stream (e.g.. Audio Stream. 
Organization Stream, Control Stream, etc.) Field 204 contains an Elementary Str^ 
Identifier (ES_ID), which is used to identify the particular stream, and may be used in cases 
where multiple streams of a particular stream type may be encomitered at the same time. 
Field 207 contains a time stamp, which is used by the system to synchronize the various 
streams, including rendering of the streams. Composite Block 1 1 may, for example, keep 
track of the elapsed time fiom the commencement of rendering. Time Stamp 207 may be 
used by Composite Block 1 1 to detennine when each object is supposed to be rendered. 
Time Stamp 207 may therefore specify an elapsed time fiom commencement of rendering, 
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and Composite Block 1 1 may use that elapsed time to determine when to render the 
associated object. 

Field 205 contains a Governance Indicator. Field 206 identifies the number of 
following packets which are part of the identified stream. In each case, the relevant 
information is encoded in a binary format. For example, Field 202 might include an 
arbitrary sequence of bits which is recognized as indicating a header, and Field 203 might 
include two bits, thereby allowing encoding of four different stream types. 

Returning to FIG. 1, System 1 includes Demux 7, which accepts as input Bit Stream 
2 and routes individual streams (sometimes referred to as Elementary Streams or "ESs") to 
appropriate fimctional blocks of the system. 

Bit Stream 2 may be encoded in the format illustrated in FIG. 3. In this figure. 
Header 301 is encountered in the bit stream, with Packet 302 following, and so on through 
Packet 308. 

When Demux 7 encounters Header 301, Demux 7 identifies Header 301 as a header 
and uses the header information to identify Packets 302-305 as organization stream 
packets. Demux 7 uses this information to route these packets to Organization Block 8. 
Demux 7 handles Header 306 in a similar mamier, using the contained information to route 
Packets 307 and 308 to AV ("Audio Video") Block 9. 

AV Block 9 includes Decompressor 10, which accepts Elementary Streams from 
Audio Stream 4 and Video Stream 5 and decompresses those streams. As decompressed, 
the stream information is placed in a format which allows it to be manipulated and output 
(through a video display, speakers, etc.). If multiple streams exist (e.g.. two video streams 
each describing an aspect of a video sequence). AV Block 9 uses the ES_ID to assign each 
packet to the ^propriate stream. 

Organization Block 8 stores pointer information identifying particular audio 
streams and video streams contained in a particular object, as well as metadata information 
describing, for example, where the object is located, when it is to be displayed (e.g.. the 
time stamp associated with the object), and its relationship to other objects (e.g., is one 
video object in fiont of or behind another video object). This organization may be 
maintained hierarchically, with individual streams represented at the lowest level, 
groupings of streams into objects at a higher level, complete scenes at a still higher level, 
and the entire work at the highest level. 

FIG. 4 illustrates one manner in which Organization Block 8 may store a 
representation of a work. In this Figure. Tree 401 represents an entire audiovisual work. 
Branch 402 represents a high-level organization of the work. This may include, for 
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example, all of the video or possibly the audio and video associated with a particular scene. 

Sub-Branch 403 represents a group of related video objects. Each such object may 
include an entire screen, or an individual entity within the screen. For example, Sub- 
Branch 403 may represent a background which does not change significantly &om one shot 
to the next. If the video is moving between two points of reference (e.g.. a conversation, 
with the camera point of view changing from one face to the other), Sub-Branch 404 could 
represent a second background, used in the second point of view. 

Nodes 405 and 406 may represent particular video objects contained within the 
related group. Node 405 could, for example, represent a distant mountain range, while 
Node 406 represents a tree immediately behind one of the characters. 

Each of the nodes specifies or contains a particular ES_ID. representing the stream 
containing the information used by that node. Node 405, for example, contains ES_ID 407, 
which identifies a particular video stream which contains compressed (and possibly 
encrypted) digital information representing the mountain range. 

Composite Block 11 accepts input from Organization Block 8 and from AV Block 
9. Composite Block 11 uses the input from Organization Block 8 to determine which 
specific audiovisual elements will be needed at any given time, and to determine the 
organization and relationship of those elements. Composite Block 1 1 accepts 
decompressed audiovisual objects from AV Block 9, and organizes those objects as 
specified by infomiation from Organization Block 8. Composite Block 1 1 then passes the 
organized infomiation to Rendering Device 12, which might be a television screen, stereo 
speakers, etc. 

Control Block 13 stores control messages which may be received through Control 
Stream 6 and/or may be watermarked into or steganographically encoded in other streams, 
including Audio Stream 4 and Video Stream 5. One confrol message fomiat is illustrated' 
by FIG. 5, which shows Control Message 501. Control Message 501 is made up of Header 
502 and Message 503. Header 502 consists of Field 508. which includes a bit pattern 
identifying the following information as a header; Stream Type Field 509, which identifies 
this as a header for the organization stream; ID Field 504. which identifies this particular 
control message; Pointer Field 505, which identifies those ESs which are controlled by this 
message; Time Stamp Field 507, which identifies the particular portion of the stream which 
is controlled by this control message (this may indicate that the entirety of the stream is 
controUed); and Length Field 506, which specifies the length (m bytes) of Message 503. 
Message 503 may include packets following Header 502. using the general format shown 
m FIG. 3. In the example shown. Control Message 501 carries the unique ID 1 1 1000, 
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encoded in ID Field 504. This control message controls ESs 14 and 95. as indicated by 
Pointer Field 505. The associated Message contains 1.024 bytes, as indicated by Length 
Field 506. 

In an alternate embodiment, the association of control to content may be made in 
Organization Block 8. which may store a pointer to particular control messages along with 
the metadata associated widi streams, objects, etc. This may be disadvantageous, however, 
in that it may be desirable to protect this association fiom discovery or tampering by users.' 
Since Control Block 13 will generally have to be protected in any event, storing the 
association in this block may make protection of Organization Block 8 less necessary. 

Control Block 13 implements control over System 1 through Control Lines 14, 15 
and 16, which control aspects of Organization Block 8, AV Block 9 and Composite Block 
1 1 , respectively. Each of these Control Lines may allow two-way communication. 

Control Lines 14 and 15 are shown as communicating with AV Block Stream Flow 
Controller 18 and with Organization Block Stream Flow Controller 1 7. These Stream 
Flow Controllers contain functionality controlled by Control Block 13. In the embodiment 
iUustrated. the Stream Flow Controllers are shown as the first stage in a two-stage pipeline, 
with information being processed by the Stream Flow Controller and then passed on to the 
associated functional block. This aUows isolation of the control functionality from the 
content manipulation and display functionality of the system, and allows control to be 
added in without altering the underlying functionality of the blocks. In an alternate 
embodiment, the Stream Flow Controllers might be integrated directly into the associated 
functional blocks. 

Stream Flow ControUen 17 and 18 contain Cryptographic Engines 19 and 20, 
respectively. These Cryptographic Engines operate under control of Control Block 13 to 
decrypt and/or cryptographicaUy vaUdate (e.g.. perform secure hashing, message 
authentication code, and/or digital signature functions) the encrypted packet streams 
received fiom Demux 7. Decryption and validation may be selective or optional according 
to the protection requirements for the stream. 

Cryptographic Engines 19 and 20 may be relatively complex, and may, for 
example, include a validation calculator that performs cryptographic hashing, message 
authentication code calculation, and/or other cryptographic validation processes. In 
addition, as is described further below, additional types of governance-related processing 
may also be used, fa one alternative embodiment, a single Stream Flow Controller may be 
used for both Organization Stream 3 and Audio/Video Streams 4-5. This may reduce the 
cost of and space used by System 1. These reductions may be significant, since System 1 
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may contain multiple AV Blocks, each handling a separate Audio or Video Stream in 
parallel. This alternative may, however, impose a latency overhead which may be 
unacceptable in a real-time system. 

If the Sb^ Flow Controllers are concentrated in a single block, they may be 
mcorporated directly mto Demux 7, which may handle governance processing prior to 
routing streams to the functional blocks. Such an embodiment would allow for governed 
decryption or validation of the entirety of Bit Stream 2. which could occur prior to the 
routing of streams to individual functional blocks. Encryption of the entirety of Bit Stream 
2 (as opposed to mdividual encryption of individual ESs) might be difficult or impossible 
without incorporating stream controller functionality into Demux 7. since Demux 7 might 
otiierwise have no abiUty to detect or read the header information necessary to route 
sti-eams to functional blocks (that header information presmnably being encrypted). 

As is noted above, each of the individual streams contained in Bit Stream 2 may be 
individually encrypted. An encrypted stream may be identified by a particular indicator in 
the header of the stream, shown in FIG. 2 as Governance Indicator 205. 

When a header is passed by Demux 7 to the appropriate functional block, the stream 
flow controller associated with that block reads the header and determines whether the 
following packets are encrypted or otherwise subject to governance. If the header indicates 
that no governance is used, the stream flow contioUer passes the header and die packets 
through to the functional blocks with no alteration. Governance Indicator 205 may be 
designed so that conventionally encoded content (e.g., unprotected MPEG-4 content) is 
recognized as having no Govemance Indicator and therefore passed through for normal 
processing. 

If a stream flow contix)ller detects a set govemance indicator, it passes the ES_ID 
associated with that stream and tiie time stamp associated with the current packets to 
Control Block 13 along Control Line 14 or 15. Control Block 13 then uses the ES_ID and 
time stamp information to identify which control message(s) are associated with that ES. 
Associated messages are then invoked and possibly processed, as may be used for 
governance purposes. 

A simple govemance case is illustrated by FIG. 6. which shows steps which take 
place using the functional blocks of FIG. 1. In Step 601, Demux 7 encounters a header, 
and determines that the header is part of the AV sti^. In Step 602, Demux 7 passes L 
header to AV Stream Controller 18. InStep603,AV Stream Controller 18 reads the 
header and determines that the govemance indicator is set, thereby triggering further 
processing along Path 604. In Step 605. AV Stream Controller 18 obtains tire ES_ID and 
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mV7 !r *° --"--i'^ -heso » Co„^, Bloc. .3. along Contro, Un. 

IS. In Step 606. Conm>l Block 13 l«,ks ^, .he ES.ID and detennfaes tha. U,e ES ID is 
a^ociaW »i.h a particular conttol n^ssage. L, Step 6, 1. ConW Block 13 uses to dme 
suunp .nfo^ation ,o choose among c„„m messages, if is more one c„„«„l 
message associaWwiftaparticularES. In S.ep 607. Control Block 13 accesses tte 
appro,»,ate conuol message. ^ obuuns a cyplographic key or keys for decyption and/or 
va^daoaln Step 608. Con.ro, Block 13 

Lme 15 10 AV Steun Consoller 18. In Slep 609. AV S«eam Conttoller 18 uses 4e 
cwu>^phic key as an input ,o Cryptographic Engine 20. which dcoypts and/or vaUda.es 
*e packe.s following a,e header as *ose packed are receiv«l fcm Demu, 7. ,n S.ep 610 
*. decop.ed packMs are passed .0 AV Block 9, which decomp^sses and processes ' 

tnem m a conventional manner. 

Time stamp infonnation may be useful when it is desirable to change the control 
message applicable toaparticularES. For example, it may be useful to encode different 
portions of a stream with different keys, so that an attacker breaking one key (or even a 
number of keys) will not be able to use the content. This can be done by associating a 
nmnber of control messages with the same stream, with each control message being valid 
for a particular period. The time stamp information would then be used to choose which 
control message (and key) to use at a particular time. Alternatively, one control message 
may be used, but with updated information being passed in through the Control Stream the 
updates consisting of a new time stamp and a new key. 

In an alternative embodiment. Control Block 13 may proactively send the 

appropriate keys to the appropriate stream flow controller by using time stamp irf^^^^^ 
to detemune when a key wiU be will be needed. TOs may reduce overall latency 

Control Line 16 from FIG. 1 comes into play once information has been passed 

from OrganizationBlock8andAVBlock9toCompositeBIockll.and the finished work 
IS pr^ared for rendering through Rendering Device 12. When Composite Block 1 1 sends 

^object toRenderingDevicelUCompositeBlockllsendsastart message to Control 
Block 13. This message identifies the object (including any associated ES IDs) and 
sp«:.fiesthestarttimeofthedisplay (or other rendering) of that object. When In object is 
no longer bemg rendered. Composite Block 1 1 sends an end message to Control Block 13 
specafymg that rendering of the object has ended, and the time at which the ending ' 
occurred. Multiple copies of a particular object may be rendered at the same time. Forthis 
reason, start and stop messages sent by Composite Block 1 1 may include an assigned 
mstance ED, which specifies which instance of an object is being rendered 
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Control Block 13 may store information relating to start and stop times of particular 
objects, and/or may pass this information to external devices (e.g.. External Server 30) 
through Port 21. This information allows Control Block 13 to keep track not only of which 
objects have been decrypted, but of which objects have actually been used. Tins may be 
used, since System I may decrypt, validate, and/or decompress many more objects than are 
actually used. Control Block 13 can also determine the length of use of objects, and can 
detennine which objects have been used together. Information of this type may be used for 
sophisticated billing and auditing systems, which are described further below. 

Control Line 16 may also be used to control the operation of Composite Block 1 1 
In particular. Control Block 13 may store infomiation specifying when rendering of a 
parhcular object is valid, and may keep track of the number of times an object has been 
rendered. If Control Block 13 detemrines that an object is being rendered iUegaUy (i e in 
violation of rules controlling rendering). Control Block 13 may terminate operation of ' 
Composite Block 1 1, or may force erasure of the illegal object. 

In an alternate embodiment, the level of contn>l provided by Control Une 16 may at 
least m part be provided without requiring the presence of that line. Instead, Control Block 
13 may store a hash of the organization information currently vaUd for Organization Block 
8. This hash may be received through Control Stream 6. or, alternatively, may be 
generated by Control Block 13 based on the information contained in Organization Block 
8. 

Control Block 13 may periodically create a hash of the information currently 
resident in Organization Block 8. and compare that to the stored hasl. A difference may 
indicate that an unauthorized alteration has been made to the information in Organization 

Block 8. thereby potentially aUowingauser to render information inamamierviolativeof 
the rules associated with that infomiation. In such an event. Control Block 13 may take 
appropriate action, including deleting the infomiation currently resident in Organization 
Block 8. 

If System 1 is designed so that Organization Block 8 controls the use of content by 
Composite Block 1 1. so that content camiot be rendered except as is specified by the 
organization information. Control Block 13 may be able to contiol rendering of 
infomiation through verifying that the current Organization Block contents match flie hash 
which has been received by Control Block 13, thereby eliminating at least one reason for 
the presence of Contiiol Line 16. 

Control Block 13 may also be responsible for securely validating the origm 
integrity, authenticity, or other properties of received content, through cryptographic 
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validation means such as secure hashing, message authentication codes, and/or digital 
signatures. 

System 1 may also include an Inter-Rights Point, indicated as IRP 22. IRP 22 is a 
protected processing environment (e.g., a PPE) in which rules/controls may be processed, 
5 and which may store sensitive information, such as cryptographic keys. IRP 22 may be 

incorporated within Control-Block 13. or may be a separate module. As is iUustrated, IRP 
22 may include CPU 23 (which can be any type of processing unit). Cryptographic Engine 
24. Random Number Generator 25, Real Time Clock 26. and Secure Memory 27. In 
particular embodiments, some of these elements may be omitted, and additional 
1 0 functionality may be included. 

Governance Rules 

Control messages stored by Control Block 13 may be very complex. FIG. 7 
illustrates the form in which the control messages may be stored in Control Block 13, 
consisting of Array 717. Column 701 consists of the address at which the control mwsages 
15 are stored. Column 702 consists of the identifier for each control message. This function 

may be combined with that of Column 701, by using the location information of Column 
701 as the identifier, or by storing the message in a location which corresponds to the 
identifier. Column 703 consists of the ES_IDs for each stream controlled by the control 
message. Column 704 consists ofthe message itself Thus, the control message stored at 
20 location 1 has the ED 1 5, and controls sti-eam 903. 

hi a simple case, the message may include a cryptographic key, used to decrypt the 
content associated with the stream(s) conh-oUed by the message. This is illustrated by 
Cryptographic Key 705 from FIG. 7. Ciyptographic keys and/or validation values may 
also be included to permit cryptographic validation ofthe integrity or origin ofthe stream. 

In a more complex case, the message may include one or more rules designed to 
govern access to or use of governed content. Rules may fell into a number of categories. 

Rules may require that a particular aspect of System 1, or a user of System l.be 
verified prior to decryption or use ofthe governed content For example. System 1 may 
include System ID 28, which stores a unique identifier for the system. A particular rule 
contained in a control message may sqjecify that a particular stream can only be decrypted 
on a system in which System ED 28 contains a particular value. This is iUustrated at row 2 
in FIG. 7. in which the message is shown as consisting of a rule and commands. The rule 
may be implicit, and therefore may not be stored explicitly in the table (e.g. die table may 
store only the rule, the rule - specific fimctions (commands) invoked by the rule, or only 
35 the functions). 
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In this case, when Stream Controller 1 8 encounters a Header for stream 203 1 
containing a set governance indicator, Stream Controller 18 passes the associated ES_ID 
(2031) to Control Block 13. Control Block 13 then uses the ES_ID to identify Control 
Message 20 which governs stream 2031. Control Message 20 includes Rule 706. which 
includes (or invokes) Commands 707, and an Authorized System ID 708. Authorized 
System ID 708 may have been received by System 1. either as part of Control Message 20. 
or as part of another control message (e.g., Control Message 9). which Control Message 20 
could then reference in order to obtain access to the Authorized System ID. Such a case 
might exist, for example, if a cable subscriber had pre-registered for a premium show. TTie 
cable system might recognize that registration, and authorize the user to view the show, by 
sending to the user an ID corresponding to the System ID. 

When Rule 706 is invoked, corresponding Commands 707 access System ID 28 and 
obtain the system ID number. The commands then compare that number to Authorized 
System ID 708, specified by Rule 706. If the two numbers match. Commands 707 release 
Cryptographic Key 709 to Stream Controller 18, which uses Cryptographic Key 709 to 
decrypt the stream corresponding to ES_ID 203 1. If the two numbers do not match. 
Commands 707 fail to release Cryptographic Key 709, so diat Stream Controller is'is 
unable to decrypt the stream. 

In order to carry out these fimctions, in one embodiment. Control Block 13 
includes, or has access to. a processing unit and memory. The processing miit is preferably 
capable of executing any of the commands which may be included or invoked by any of the 
rules. The memory will store the rules and association infonnation (ID of the control 
message and IDs of any governed ESs). 

Since the functions being carried out by Control Block 13 are sensitive, and involve 
govemance of content which may be valuable. Control Block 13 may be partiaUy or 
completely protected by a barrier which resists tampering and observation. As is described 
above, the processing unit, secure memory, and various other govemanc^related elements 
may be contained in IRP 22. which may be included in or separate fiom Control Block 1 3. 

Control Block 13 may also carry out somewhat more complex operations. In one 
example, a control message may require that infonnation from System 1 not only be 
accessed and compared to expected infomiation, but stored for future use. For example, a 
control message might allow decryption of a Stream, but only after System ID 28 has b^ 
downloaded to and stored in Control Block 13. This would allow a control message to 
check the stored System ID against System ID 28 on a regular basis, or perhaps on every 
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attempted re-viewing of a particular Stream, thereby allowing the control message to insure 
that the Stream is only played on a single System. 

Control Block 13 may also obtain information dynamicaUy. For example, System 1 
may include User Interface 29, which can include any type of user input fimctionaUty (e.g.. 
hardware buttons, information displayed on a video saeen, etc.) A particular rule from a ' 
control message may require that the user enter information prior to allowing decryption or 
use of a stream. That information may, for example, be a password, which the Rule can 
then check against a stored password to insure that the particular user is authorized to 
render the stream. 

Information obtained from the user might be more complicated. For example, a 
rule might require that the user input payment or personal information prior to allowing 
release of a cryptographic key. Payment information could, for example, constitute a credit 
card or debit card number. Personal information could include the user's name, age, 
address, email address, phone number, etc. Entered information could then be sent through 
Port 21 to External Server 30 for verification. Following receipt of a verification message 
from External Server 30, the Rule could then authorize release of a cryptographic key. 
Alternatively, Control Block 13 may be designed to operate in an "ofF-line" mode, storing 
the information pending later hookup to an external device (or network). In such a case. 
Control Block 13 might require that a connection be made at periodic intervals, or might 
limit the number of authorizations which may be obtained pending the establishment of an 
extemal coimection. 

In a somewhat more complex scenario, a control message may include conditional 
rules. One particular example is iUustrated by row 4 of the table shown in FIG. 7, in which 
Control Message 700 is shown as controlling streams 49-53. Control Message 700 fiirther 
consists of Rule 710, Commands 71 1 and Cryptographic Keys 712-716. TTiere could, of 
course, be a number of additional cryptographic keys stored with the message. 

In this case. Rule 710 specifies that a user who agrees to pay a certain amount (or 
provide a certain amount of infonnation) may view Stream 49, but all other users are 
required to view Stream 50. or a combination of Streams 49 and 50. In this case. Stream 
49 may represent a movie or television program, while Stream 50 represents 
advertisements. In one embodiment, different portions of Stream 49 may be decrypted 
with different keys so that, for example, a first portion is decrypted with Key 712, a second 
portion is deciypted with Key 713. a third portion is decrypted with Key 714, and so on. 
Rule 710 may include all keys used to decrypt the entirety of Stream 49. When the user 
initially attempts to access the video encoded in Stream 49, Rule 710 could put up a 
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message asking if the user would prefer to use pay for view mode or advertising mode If 
the user selects pay for view mode. Rule 7 1 0 could store (or transmit) the payment 
mfonnation, and pass Cryptographic Key 712 to Stream Controller 18. Stream ControUer 
18 could use Cryptographic Key 712 to decrypt the first stream until receipt of a header 
indicating that a different key is needed to decrypt the following set of packets Upon 
request by Stream Controller 18. Control Block 13 would then check to detemune that 
payment had been made, and then release Cryptographic Key 713. which would be used to 
decrypt the foUowing packets, and so on. Rule 710 could additionally release 
Cryptographic Key 716. conespondmg to Organization Stream 52. which corresponds to 
video without advertisements. 

If. on the other hand, the user had chosen the advertising mode. Rule 710 could 
release Cryptographic Key 712 to Stream Controller 18 to allow dec^rption of Stream 49 
Rule 710 could also authorize decryption of Stream 50 which contains the advertisements 
Rule 710 could further release Cryptographic Key 715 to Organization Block 8 
Cryptographic Key 715 matches Organization Stream 51. Organization Stream 51 
references the video from Stream 49. but also references advertisements from Stream 50 
Rule 710 would refuse to release Cryptographic Key 716, which corresponds to 
Organization Stream 52. which corresponds to the video without advertisements. 

In operation. Control Block 13 could monitor infomiation from Composite Block 
1 1 over Control Line 16. Ttat information could include the identity of each object 
actually rendered, as well as a start and stop time for the rendering. Control Block 13 
could use this information to determine that an advertisement had actually been rendered 
pnortoreleasmgCryptographicKey713 for decryption ofthe second portion of video ' 
from Stream 49. This feedback loop allows Control Block 13 to be certain that tiie 
advertisements are not only being decrypted, but are also being displayed. TOs may be 
necessary because CompositeBlock 11 may be relatively unprotected, thereby allowing an 
unscrupulous user to remove advertisements before viewing. 

A variety of additional relatively complex scenarios are possible. For example 
rules from Control Block 13 could customize tiie programming for a particular geographic 
location or a particular type of viewer, by using information on the location or the viewer 
to conbrol conditional decryption or use. This information could be stored in System 1 or 
entered by the user. 

In another example, shown at row 5 of Array 717. Rule 719 may specify Budget 
718, which may include infomiation relating to tiie number of uses available to the user 
the amount of money tiie user has to spend, etc. In operation. Rule 719 may require thai 
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Budget 71 8 be securely stored and decremented each time a budgeted activity occurs (e.g.. 
each time the associated work is played). Once the budget reaches zero. Rule 719 may 
specify that the work may no longer be played, or may display a message to the user 
indicating that the user may obtain additional budget by. for example, entering a credit card 
number or password, or contacting an external server. 

In another example, a rule may control the ability of a user to copy a work to 
another device. The rule may. for example, specify that the user is authorized to use the 
govemed woric on more than one device, but with only one use being valid at any time. 
The rule may specify that an indication be securely stored regarding whether the user has 
"checked out" the work. If the user copies the work to another device (e.g., through Port 
21). tiie rule may require that the work only be ti^smitted in encrypted fora,. and that the 
relevant control messages be transmitted along with it. The rule can further require that an 
mdicator be securefy set. and Uiat the indicator be checked each time the user attempts to 
use or copy the work. If the indicator is set. the rule might require that the work not be 
decrypted or used, since the user only has the right to use the work on one device at a time 
and Uie indicator establishes that the work is currently "checked out" to another device and 
has not been checked back in. 

The receiving device may include the same type of indicator, and may allow the 
user to use the work only as long as the indicator is not set. If the user desires to use the 
work on tiie original device, tire two devices may communicate, with U,e indicator being set 
m tiie second and reset in tiie first. TTiis allows tiie work to be stored in two locations, but 
only used in one. 

In another embodunent, tiie same result may be reached by copying the relevant 
control message from one device to the odier. then erasing it from the original device. 
Because the control message includes keys used for decryption, this would insure that the 
work could only be used in one device at a time. 

In one embodiment, this technique may be used to communicate digital media files 
(e.g.. music, video, etc.) from a personal computer to a consumer electronics device 
without aUowing the user to make multiple choices for simultaneous use. TTrus. a larger, 
more sophisticated device (e.g., a personal computer), could download a file, then "check 
out" Uie file to a portable device lacking certain fimctions present in the personal computer 
(e.g., a hand-held music player). 

Rules may also be used to specify that an initial user may transfer the file to another 
user, but only by giving up control over the file. Such rules could operate similarly to the 
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technique described above for transferring a file from one device to another, or could 
require that the original file be entirely erased from the original device after the transfer. 

Rules in Control Block 13 may be added or updated through at least two channels 
New rules may be obtained through Control Stream 6. If a control message contains an 
identifier corresponding to a control message already present in Control Block 13, that 
control message (including contained rules) may overwrite the original control message A 
new rule may. for example, be identical to an existing rule, but with a new time stamp and 
new keys, thereby allowing decryption of a stream which had been encrypted with multiple 
keys. System 1 may be designed so that certain rales may not be overwritable. This may 
be enforced by designating certain positions in Array 71 7 as non-overwritable, or by 
providing a flag or other indicator to show that a particular rule camiot be ovewritten or 
altered. This would allow for certain types of superdistribution models, including allowing 
a downstream distributor to add rules without allowing the downstream distributor to 
remove or alter the rales added by upstream distributors. 

In addition, new rules may be encoded into Organization Stream 3. Audio Stream 4. 
or Video Stream 5, in the form of a watermaric or steganographic encoding. 

New rules may also be obtained through Port 21. Port 21 may connect to an 
external device (e.g., a smart card, portable memory, etc.) or may comiect to an external 
networic (e.g., External Server 30). Rules may be obtained through Port 21 either in an ad 
hoc manner, or as a result of requests sent by Control Block 13. For example, Control 
Message 14 (FIG. 7. row 6) may mclude a rule specifying that a new mle be downloaded 
from a particular URL, and used to govern Stream 120 1 . 

Control messages, including rales, may be encoded using secure transmission 
formats such as DigiBoxes. A DigiBox is a secure container means for delivering a set of 
business rules, content description information, content decryption information and/or 
content validation information. One or more DigiBoxes can be placed into the headers of 
the media content or into data streams within the media 

FIG. 12 illustrates one embodiment of the DigiBox format and the mamier in which 
that format is incorporated into a control message. Control Message 1201 is made up of 
Contiol Message Header 1202 and Control Message Contents 1203. As is described 
elsewhere. Control Message Header 1202 may include information used by Demux 7 (FIG. 
1) to ^propriately route tiie message to Control Block 13. 

Control Message Contents 1203 of Control Message 1201 consists of DigiBox 
1204. and may also include additional infonnation. DigiBox 1204 consists of DigiBox 
Header 1205. Rules 1206 and Data 1207. Rules 1206 may include one or more rules. Data 
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1207 may include various types of data, incb^^ 1208. Cryptographic Key 1209. 
and Va^^dat^on Data 1 2 10. Data 1207 may also include cryptographic informad^ 
a specification of the encryption algorithm, chaining modes used with the algorithm keys 
and mitialization vectors used by the deciyption and chaining. 

^«^i^«onvectorscontainedwithinDatal207aresimilartocryptographickeys 
m that they constitute input to the original encryption process and therefore are necessary ' 
for decryptioa In one well-known prior art embodiment, the initialization vector, may be 

generated by starting withabase initialization vector(a64 bit random number) and xor'ing 
m the frame number or start time for the content item. 

VaUdation Data 1210 contained within Data 1207 may include cryptographic has or 
authentication values, cryptographic keys for calculating keyed authentication values (e g 
message authentication codes), digital signatures, and/or public key certificates used in " " 
validating digital certificates. 

Thus, the DigiBox may incorporate the infomiation described above as part of the 
conti-ol message, includmg the rules, the st^ ID and the cryptographic keys and values 

In an alternative embodiment, DigiBox Header 1205 may be designed so that it can 
be read by Demux 7 and routed to Control Block 13. In such an embodiment. DigiBox 
1204 would itself constitute the entirety of tiie control message, thus obviating the need to 
nest DigiBox 1204 within Control Message 1201. 

Some or all of the contents of DigiBox 1204 will genemlly be encrypted. TTiis may 
mclude Rules 1206. Data 1207. and possibly some or all of Header 1205. System 1 may be 
designed so that a DigiBox may only be decrypt«l (opened) in a protected environment 
such as IRP 22. In an alternate embodiment. Control Block 13 may directly incorporate tiie 
fimctionaUty of IRP 22. so that the DigiBox may be opened in Control Block 13 without 
thenecessity ofroutingthe DigiBox to IRP 22 forprocessing. In one embodiment the 
cryptographic key used to decrypt DigiBox 1204 may be stored in IRP 22 (or Control 
Block 13). so that the DigiBox can only be opened in that pn,tected enviromnent. 

Rules 1206 are rales governing access to or use of DigiBox Data 1207 In one 
embodiment, these mles do not directly control the governed streams. Since Cryptographic 
Key 1209 can only be accessed and used through compUance with Rules 1206. however 
Rules 1206 in fiict mdkectly control the governed streams, smce those streams can only be 
deciypted through use of the key. which can only be obtamed in compliance with the rules 
In another embodunent. Data 1 207 may include additional rales, which may be extracted 
torn the DigiBox and stored in a table such as Array 717 of FIG. 7. 
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The rules governing access to or use of a DigiBox may accompany the DigiBox (as 
shown m FIG. 12) or may be separately transmitted, in which event Rules 1206 would 
contain a pointer or reference to the rules used to access Data 1207. Upon receipt of a 
D.giBox, Control Block 1 3 may receive rules separately through Control Stream 6. or may 
request and receive rules through Port 21. 

Pipelined Implementation 

One potential drawback to the system illustrated in FIG. 1 consists of the fact that 
the system introduces complexity and feedback into a pipeUned system designed to render 
content in real tune. ITie rendering pipeline generally consists of Demux 7. Organization 
Block8andAVBlock9.CompositeBlock 11 and Rendering Device 12. Because content 
.s received in a streamed fashion, and must be rendered in real time, pipelined processing 
must occur in a highly efficient manner, under tight tune constraints. A failure to process 
wxthm the time available may mean that output to Rendering Device 12 may be interrupted 
or that incoming Bit Stream 2 may overflow available buffers, tiiereby causing the loss of ' 
some portion of the incoming data. 

An alternative embodiment of System 1 is designed to address these problems 
although at a possible cost in the abiUty to use standard system components and a possible 
cost m overall system security. This alternative embodiment is illustrated in HG. 1 1 
which shows System 1 101. 

System 1 101 is similar to System I from na 1 in many respects. It receives Bit 
Stream 1 102, which consists of Organization Stream 1 103. Audio Stream 1 104 Video 
Stream 1 105 and Control Stream 1 106. These streams are received by Demux 1 107. which 
passes Organization Stream 1103 to Organization Block and passes Audio Stream 1104 
and Video Stream 1105 to AVBlock 1109. Organization Block 1 108 and AV Block 1 109 
operate similarly to their counterparts in FIG. 1. and pass information to Composite Block 
1110. which organizes the infomiation into a coherent whole and passes it to Rendering 
Device nil. Streams sent to Organization Block 1108 are decrypted and/or validated by 
Stream Flow Controller 1 1 12. and streams sent to AV Block 1 109 are decrypted and/or 
validated by Stream Flow Controller 1113. 

System 1101 differs fiom System I, however, in that control and feedback are 
distnbuted, and integrated directiy into the processing and rendering pipelme. System 
1 101 thus lacks a separate control block, and also lacks a feedback path back fiom the 
Composite Block 1110. 

In System 1101. control is exercised directly at Organization Block 1 108 and AV 
Block 1109. AsinSystem L cryptographic keys are received through Control Stream 1106 
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( in an altenuitive embodiment, the keys could be incorporated direcUy into header or other 
infonnation in Organization Stream 1 103 or Audio/Video Streams 1 104 and 1 105). Those 
keys are included in a data format which includes information regarding the stream type of 
the encrypted content and. if multiple stream types are possible, an identifier for the 
particular controlled stream. 

When Demux-1 107 encounters a key in Control Stream 1 106, it reads the 
information relating to the stream type, and routes the key to the appropriate stream flow 
controUer. IfDemux HO? encounters a key designated for decryption or validation of 
Organization Stream 1 103, for example, it routes that key to Stream Flow Controller 1112. 

Stream Flow ControUer 1112 stores received keys in Storage Location 1114. 
Storage Location 1114 stores the keys and also stores an indicator of the controlled stream 



ID. 



Stream Flow Controller 1112 includes Cryptographic Engine 1115, which uses the 
received keys to decrypt and/or validate encrypted and/or protected portions of 
Organization Stream 1103. The keys may themselves be received in an encrypted mamier, 
in order to provide some degree of security. In such a case. Stream Flow ControUer may ' 
use a variety of techniques to decrypt the key. including using stored information as a key, 
or as a key seed. That stored information could, for example, constitute a "meta-key" 
provided eariier through Bit Stream 1 102 or through a separate port 

Stream Flow Controller 1 1 13, associated with AV Block 1 109. contains a 
corresponding Storage Location 1 1 16 and Cryptographic Engine 1 1 17. and operates in a 
manner similar to the operation described for Stream Flow Controller 1 1 12. 

TTiis implementation avoids the latency penalty which may be inherent in the 
necessity for commmiication between stream flow controllers and a separate control block. 

This alternate implementation may also eUminate the feedback channel fiom the 
composite block (FIG.l. Control Line 16). This feedback chamiel may be used in order to 
insure that the content being passed fiom Composite Block 1 1 to Rendering Device 12 is 
content that has been authorized for rendering. In the alternate embodiment shown in 
nai 1, this feedback chamiel does not exist Instead, this implementation reUes on the 
fact that Composite Block 1110 depends upon information fiom Organization Block 1 1 08 
to determine the exact stmctiire of the information being sent to Rendering Device 1111. 
Composite Block 1 1 10 cannot composite infonnation in a manner contrary to the 
organization dictated by Organization Block 1 108. 

In one embodiment, this control by Organization Block 1 108 may be sufficient to 
obviate the need for any feedback, since Organization Block 1 1 08 may be designed so that 
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it accepts infonnation only through SHeam Ccnttoller 11 12, and Stream Controller 1 1 12 
may be designed so that it only decrypts or validates infonnation under the control of rules 
Stored in Storage Location 1114. 

In such an embodiment, security may be further increased by incorporating Secure 
Memory 1 118 into Organization Block 1 108. Secure Memory 1 1 18 may store a copy or 
hash of the organization tree validly decrypted by Stream Controller 1 1 12, and in current 
use m Main Organization Block Memory 1 1 19. Organization Block 1 108 may be used to 
penodically compare the organization tree stored in Main Organization Block Memory 
1 1 19 to the tree stored in Secure Memory 1 1 18. If a discrepancy is spotted, this may 
mdicate that an attacker has altered the organization tr«e stored in Main Organization Block 
1.1 19, thereby possibly allowing for the rendering of content in violation of applicable 
rules. Under such circumstances. Organization Block 1 1 08 may be used to take protective 
measures, including replacing the contents of Main Organization Block Memory 1 1 1 9 with 
the contents of Secure Memory 1118. 

MPEG-4 Implementation 

The generic system described above may be embodied in an MPEG-4 system, as 
illustrated in FIG. 8, which shows MPEG-4 System 801. 

MPEG-4 System 801 accepts MPEG-4 Bit Stream 802 as input. MPEG-4 Bit 
Stream 802 includes BIFS Stream 803. OD Stream 804. Audio Stream 805. Video Stream 
806 and IPMP Stream 807. These streams are passed to Demux 808. which examines 
header information and routes packets as appropriate, to BIFS 809, AVO 810 OD 81 1 or 
IPMP System 812. 

IPMP System 812 receives IPMP messages through IPMP Stream 807 Those 
messages may inctode header infonnation identifying the particular message, as well as an 
associated IPMP message. TTie IPMP message may include control infomiation, which 
may mclude a cryptographic key, validation infomiation. and/or may include complex 
governance rales, as are described above. 

Stream CorrtroUers 813. 8 14 and 815 act to decrypt, validate, and/or govern streams 
passed to BIFS 809, AVO 810 and OD 81 1. respectively. 

OD 81 1 holds object descriptors, which contain metadata describing particular 
objects, nris metadata includes an identifier of the particular Elementary Stream or 
streams which include the object, and may also inchide a pointer to a particular IPMP 
message which governs the object. Altematively. the relationship between IPMP messages 
and particular objects or streams may be stored in a table or other fonn within IPMP 
System 812. 



SUBSTITUTE SHEET (RULE 26) 



'^<*99,m96 PCTyUS99/05734 

-22- 

IPMP System 812 may exercise control over other functional blocks through 
Control Lines 816, 817, 818 and 819, each of which may transmit control/governance 
signals from IPMP System 812 and infomiation or requests fiom other functional blocks to 
IPMP System 812. The infonnation requests may mclude an ES_ID and a time stamp, 
which IPMP System 812 may use to detemune which particular message (e.g.. key) should 
be used and when. 

In an alternative embodiment, IPMP System 812 may exercise control over 
Composite and Render 821 by receiving a hash of the currently valid BIFS tree (possibly 
through IPMP stream 807), and periodicaUy checking the hash against the BIFS tree stored 
in BIFS 809. Because BIFS 809 controls die mamier in which Composite and Render 821 
renders infomiation. if IPMP System 812 conlimis that the current BIFS tree is the same as 
the authorized tree received through BIFS Stream 803. IPMP System 812 can confirm that 
the proper content is being rendered, even without receiving feedback directly from 
Composite and Render 821. This may be necessary, since BIFS 809 may communicate 
with Port 822, which may allow a user to insert information into BIFS 809. thereby 
creating a possibility that a user could insert an unauthorized BIFS tree and thereby gain 
unauthorized access to content. 

When a stream controller receives encrypted or otherwise governed infomiation, it 
may send the ES_ID and time stamp directly to IPMP System 812. Alternatively, it may 
send this infomiation to OD 81 1 . which may reply with the ID of the IPMP message which 
governs that object or stream. The stream controller can then use that IPMP message ID to 
request decryption, validation, and/or governance fiom PMP System 812. Alternatively, 

0D811 canpasstheIPMPIDtoIPMPSystem812,whichcaninitiatecontactwiththe ' 
appropriate stream controller. 

IPMP System 812 may obtain IPMP infomiation through two chamiels other than 
IPMP Stream 807. The first of these chamiels is Port 820. which may be directly 
comiected to a device or memory (e.g., a smart card, a DVD disk, etc.) or to an external 
network (e.g., the Internet). An IPMP message may contain a pointer to information 
obtainable through Port 812, such as a URL. address on a DVD disk. etc. That URL may 
contain specific controls needed by the IPMP message, or may contain ancillary required 
information, such as, for example, information relating to the budget of a particular user. 

IPMP System 812 may also obtain IPMP information through OD updates 
contained in OD Stream 804. OD Stream 804 contains metadata identifying particular 
objects. A particular OD Message may take the format shown in FIG. 9. In this figure, OD 
Message 901 includes Header 902, which identifies the following packets as part of the' OD 



SUBSTITUTE SHEET (RULE 26) 



wo 99/48296 PCTAJS99/05734 

-23- 

stream,andindicatesthenumberofpackets. OD Message 901 further consists of Message 
903, which includes a series of Pointers 904 and associated Metadata 905. Each Pointer 
904 identifies aparticular Elementary Stream, and the associated metadata is applicable to 
that stream. Finally, OD Message 901 may contain an IPMP Pointer 906, which identifies 
a particular IPMP message. 

In aggregate, the information contained in OD-Message 901 constitutes an object 
descriptor, since it identifies and describes each elementary stream which makes up the 
object, and identifies the IPMP message which governs the object OD Message 901 may 
be stored in OD 81 1. along with other messages, each constitutmg an object descriptor. 

Object descriptors stored in OD 811 may be updated through OD Stream 804, 
which may pass through a new object descriptor corresponding to the same object. The 
new object descriptor then overwrites the existing object descriptor. This mechanism may 
be used to change the IPMP message which controls a particular object, by using a new 
object descriptor which is identical to the existing object descriptor, with the exception of 
the IPMP pointer. 

OD Stream 804 can also cany IPMP_DescriptorUpdate messages. Each such 
message may have the same format as IPMP messages carried on the IPMP stream, 
including an IPMP ID and an IPMP message. 

IPMP_DescriptorUpdate messages may be stored in a table or array in OD 8 1 1 , or 
may be passed to IPMP System 812, where they may overwrite existing stored IPMP ' 
messages, or may add to the stored messages. 

Since PMP mformation may be separately conveyed through the OD stream or the 
IPMP stream. MPEG-4 System 801 may be designed so that it only accepts information 
through one or the other of these channels. 

In another embodiment, the existence of the two channels may be used to aUow 
multi-stage distribution, with govemance added at later stages, but with no risk that later 
alterations may override govemance added at an earlier stage. 

Such a system is Ulustrated in nO. 10. In this Figure, IPMP System 812 includes 
IPMP Table 1002. which has slots for 256 IPMP messages. This table stores the IPMP_ID 
implicitiy, as the location at which the information is stored, shown in Column 1003. The 
IPMP message associated with IPMP_ID 4, for example, is stored at slot 4 of IPMP Table 
1002. 

Each location in IPMP Table 1002 includes Valid Indicator 1004 and Source 
hidicator 1 005. Valid Indicator 1004 is set for a particular location when an IPMP message 
is stored at that location. This allows IPMP System 812 to identify slots which are 
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unfiUed. which otherwise might be difficult, since at start-up the slots may be filled with 
random information. This also allows IPMP System 812 to identify messages which are no 
longer valid and which may be replaced. Valid Indicator 1004 may store time stamp 
information for the period during which the message is valid with IPMP System 812 
determining vaUdity by checking the stored time stamp information against the currently 
valid time. 

Source Indicator 1005 is set based on whether the associated IPMP message was 
received from IPMP Stream 807 or fiom OD Stream 804. 

These indicators allow IPMP System 812 to establish a hierarchy of messages, and 
to control the manner in which messages are added and updated. IPMP System 812 may 
be designed to evaluate the indicators for a particular location once a message is received 
corresponding to that location. If the valid indicator is set to invalid. IPMP System 812 
may be designed to automaticaUy write the IPMP message into that slot. If the valid 
indicator is set to valid, IPMP System 812 may then be designed to check the source 
indicator. If the source indicator indicates that the associated message was received 
through OD Stream 804. IPMP System 1812 may be designed to overwrite the existing 
message with the new message. If; however, the source indicator indicates tiiat the 
associated message was received through IPMP Stream 807. IPMP System 812 may be 
designed to check the source of the new message. That check may be accomplished by 
examining the header associated with the new message, to determine if the new message 
was part of OD Stream 804 or part of IPMP Stream 807. Alternatively, IPMP System 812 
may derive this information by determining whether the message was received directly 
fiom Demux 808 or through OD 8 1 1 . 

If the new message came through IPMP Stream 807, IPMP System 812 may be 
designed to store the new message in Table 1002. overwriting the existing message. If the 
new message came through OD Stream 804, on the other hand, IPMP System 8 12 may be 
designed to reject the new message. 

This message hierarchy can be used to aUow for a hierarchy of control. A studio, 
for example, may encode a movie in MPEG-4 format. The studio may store IPMP 
messages in the IPMP stream. Those messages may include a requirement that IPMP 
System 812 require that a trailer for another movie fix>m the same studio be displayed prior 
to the display of the feature movie. IPMP System 812 could be used to monitor the 
beginning and end of rendering of the trailer (using feedback through Control Line 819) to 
ensure that the entire trailer plays, and that the user does not fast-forward through it. 
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The movie studio could encrypt the various elementary streams, including the 
IPMP stream. The movie studio could then provide the movie to a distributor, such as a 
cable channel. The movie studio could provide the distributor with a key enabling the 
distributor to decrypt the OD stream (or could leave the OD stream unencrypted), and the 
abiUty to insert new messages in that stream. The cable channel could, for example, 
include a rule in the OD stream specifying^hat the IPMP system check to determine if a 
user has paid for premium viewing, decrypt the movie if premium viewing has been paid 
for. but insert advertisements (and require that they be rendered) if premium viewing has 
not been paid for). 

The cable channel would therefore have the ability to add its own rules into the 
MPEG-4 Bit Stream, but with no risk that the cable channel would eUminate or alter the 
rules used by the movie studio (e.g.. by changing the trailer from a movie being promoted 
by die studio to a rival movie being promoted by the cable channel). The studio's rules 
could specify the types of new rules which would be allowed through the OD stream. 
1 5 thereby providing the studio a high degree of control. 

This same mechanism could be used to allow superdistribution of content, possibly 
from one user to another. A user could be provided with a programming interface enabling 
the insertion of messages into the OD stream. A user might, for example, insert a message 
requiring that a payment of $ 1 .00 be made to the user's account before the movie can be 
viewed. The user could then provide the movie to another user (or distribute it through a 
medium whereby copying is uncontrolled, such as the Internet), and still receive payment. 
Because the user's rules could not overrule the studio's rules, however, the studio could be 
certain that its rules would be observed. Those might include rules specifying die types of 
rules a user would be allowed to add (e.g., limiting the price for redistribution). 

MPEG-4 System 801 may also be designed to include a particular type of IPMP 
system, which may be incompatible with EPMP systems that may be designed into other 
MPEG-4 systems. This may be possible because die MPEG-4 standard does not specify 
the format of the information contained in the IPMP stream, thereby aUowing different 
content providers to encode information in differing manners. 

IPMP System 812 m MPEG-4 System 801 may be designed for an enviromnent in 
which diflfering IPMP formats exist. That system may scan the IPMP stream for headers 
that are compatible witii IPMP System 8 12. All other headers (and associated packets) 
may be discarded. Such a mechanism would allow content providers to incorporate the 
same IPMP message in multiple fomiats, widiout any concem that encountering an 
unfamiliar format would cause an IPMP system to fail. In particular, IPMP headers can 
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incorporate an IPMP System Type Identifier. Those identifiers could be assigned by a 
central authority, to avoid the possibility that two incompatible systems might choose the 
same identifier. 

IPMP System 801 might be designed to be compatible with multiple formats. In 
such a case. IPMP System 801 might scan headers to locate the first header containing an 
IPMP System Identifier compatible with IPMP System 801 . IPMP System 801 could then 
select only headers corresponding to that IPMP System Identifier, discarding all other 
headers, including headers incorporating alternate IPMP System Identifiers also recognized 
by the IPMP system. 

Such a design would allow a content provider to provide multiple formats, and to 
order them from most to least preferred, by including the most preferred format first, the 
second most preferred format second, and so on. Since IPMP System 801 locks onto the 
first compatible format it finds, this ordering in IPMP Stream 801 would insure that the 
IPMP system chose the format most desired by the content provider. 

Even if different IPMP formats are used, content will probably be encoded (and 
encrypted) using a single algorithm, since sending multiple versions of content would 
impose a significant bandwidth burden. Thus, ordinarily it will be necessary for content to 
be encrypted using a recognized and common encryption scheme. One such scheme could 
use the DBS algorithm in output feedback mode. 

This method of screening IPMP headers, and locking onto a particular format may 
also be used to customize an MPEG-4 bit Stream for the fimctional capabilities of a 
particular MPEG-4 system. Systems capable of rendering MPEG-4 content may span a 
considerable range of fimctionality, from high-end home theaters to handheld devices. 
Governance options suitable for one type of system may be irrelevant to other systems. 

For example, MPEG-4 System 801 may include a connection to the Internet 
through Port 820, whereas a second MPEG-4 system (for example a handheld Walkman- 
like device) may lack such a connection. A content provider might want to provide an 
option to a viewer, allowing the viewer to see content for free in return for providing 
information about the viewer. The content provider could insert a rule asking the user 
30 whether the user wants to view the content at a cost, or enter identification information. 

The rule could then send the information through a port to the Internet, to a URL specified 
in the rule. A site at that URL could then evaluate the user information, and download 
advertisements targeted to the particular user. 

Although this might be a valuable option for a content provider, it obviously makes 
no sense for a device which is not necessarily connected to the Intemet It would make no 
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sense to present this option to the user of a non-connected device, since even if that user 
entered the information, the rule would have no way to provide the infonnation to an 
external URL or download the advertisements. In such a case, the content provider might 
prefer to require that the user watch preselected ads contained in the original MPEG-4 bit 
stream. 

Header infonnation in the IPMP stream could be used to customize an MPEG-4 bit 
stream for particular devices. As with the IPMP System Type inforaiation, IPMP Header 
mformation could include MPEG-4 System Types. These could include 8 or 16-bit values 
with particular features represented by bit maps. Thus, the presence of a bit at position 2 ' 
for example, could indicate that a device includes a persistent connection to the Internet. ' 

An IPMP system could then evaluate the headers, and lock on to the first header 
describing functionaUty less than or equal to the functionality contained in the MPEG-4 
device in which the IPMP system is embedded. If the header constituted a complete match 
for the fimctionality of the MPEG-4 device, the IPMP system could then cease looking If 
the header constitutes less than a complete match (e.g., a header for a system which has an 
Internet comiection, but lacks a digital output port, when the system includes both), the 
IPMP system can lock on to that header, but continue to scan for closer matches, locking 
on to a closer match if and when one is found. 

The IPMP messages identified by a particular header would be those suited for the 
particular functionality of the MPEG-4 device, and would allow for customization of the 
MPEG-4 bit stream for that functionality. In the context ofthe example given above the 
IPMP system for an MPEG-4 device containing an Internet comiection would lock on to a 
particular header, and would download the IPMP messages characterized by that header 
Those messages would prompt the user for information, would provide that information to 
the URL, and would authorize decryption and rendering ofthe movie, with the 
advertisements inserted at the appropriate spot. 

In the case of an MPEG-4 device without an Internet comiection, on the other hand, 
the IPMP system would lock onto a set of headers lacking the bit indicating an Internet 
comiection, and would download the rules associated with that header. Those rules might 
not provide any option to the user. Hie rules might allow decryption ofthe content, but 
would also specify decryption of an additional ES fiom the MPEG-4 stream. That 
additional ES would contain the advertisements, and the IPMP system would require 
decryption and rendering ofthe advertisements, checking Control Line 819 to make certain 
that this had occurred. In the case ofthe system with the Internet connection, however, the 
rules allowing deciyption and requiring rendering ofthe ES containing the advertisements 
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would never be loaded, since those rules would be contained within messages identified by 
the wrong type of header. The advertisement ES would therefore never be decrypted and 
would be ignored by the MPEG-4 device. 

no. 21 illustrates one manner in which a protected MPEG-4 file may be created. 
In this figure, CreateBox 2101 represents a DigiBox creation utility, which accepts keys 
and niles. In one embodiment, CreateBox 2101 may pass these keys and rules to IRP 2102 
and receive DigiBox 2103 fi-om IRP 2102. In another embodiment, IRP 2102 may be 
incorporated into CreateBox 2101, which accepts keys and rules and outputs DigiBox 
2103. 

DigiBox 2103 contains governance rules, initialization vectors and keys. DigiBox 
2103 is passed fi-om CreateBox 2101 to Bif Encoder 2104. Bif Encoder 2104 may be 
conventional, with the exception that it is designed to accept and process DigiBoxes such 
as DigiBox 2103. Bif Encoder 2104 also accepts a .txt file containing a scene graph, and 
initial object descriptor commands. 

Bif Encoder 2104 outputs a .bif file, containing the scene graph stream (in 
compressed binary fonn) and a .od file, containing the initial object descriptor commands, 
the object descriptor stream, and DigiBox 2103. 

Bif Encoder 2104 passes the .bif file and the .od file to Mux 2105. Mux 2105 also 
accepts compressed audio and video files, as well as a .scr file that contains the stream 
description. Mux 2105 creates IPMP streams, descriptors and messages, encrypts the 
content streams, interieaves the received streams, and outputs Protected MPEG-4 Content 
File 2106, consisting of Initial Object Descriptor 2107 and Encrypted Content 2108. Initial 
Object Descriptor 2107 contains DigiBox 2103, as well as other information. Encrypted 
Content 2108 may include a scene graph stream (i.e., a BIFS stream), an object descriptor 
25 stream, IPMP streams, and encrypted content streams. 

If DigiBox 2103 contains all keys and rules necessary to render all of the content, it 
may be umiecessary for Mux 2105 to create any IPMP streams. If additional keys or rulls 
may be necessary for at least a portion of the content. Mux 2105 may incorporate those 
rules and keys into one or more additional DigiBoxes, and incorporate those DigiBoxes 
30 either in the IPMP stream or in the OD update stream. 

FIG. 22 illustrates one manner in which control may be incorporated into an 
existmg MPEG-4 stream. In this figure. Unprotected MPEG-4 Content File 2201 includes 
Initial Object Descriptor 2202 and Content 2203. The content may include a scene 
description stream (or BIF stream), an object descriptor stream, a video stream, an audio 
35 stream, and possibly additional content streams. 
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Unprotected MPEG-4 Content File 2201 is passed to Repackager 2204. which also 
accepts keys and rules. Repackager 2204 passes the keys and rules to IRP 2205, and 
receives DigiBox 2206 in return, containing keys, rules and initialization vectors. In an 
alternate embodiment, IRP 2205 may be incorporated directly into Repackager 2204. 

Repackager 2204 demuxes Unprotected MPEG-4 Content File 2201. It inserts 
DigiBox-2206 into the Initial Object Descriptor and encrypts the various content streams. 
Repackager 2204 also adds the IPMP stream, if this is necessary (including if additional 
DigiBoxes are necessary). 

Repackager 2204 outputs Protected MPEG-4 Content File 2207, consistmg of 
Initial Object Descriptor 2208 (including DigiBox 2206) and Encrypted Content 2209 
(consisting of various streams, including the IPMP streams, if necessary). 

Real Networks Implementation 

In one embodiment, the elements described above may be used in connection with 
information encoded in compliance with formats established by Real Networks, Inc. 

The Real Networks file format (RMFF) is illustrated in FIG. 13. This format 
includes a block of headers at the beginning (Header 1301). followed by a collection of 
content packets (Content 1302), followed by an index used for seek and goto operations 
(Index 1303). Each file can contain several streams of different types. For each stream, 
there is a "Media Properties Header" (1304) used to describe the format of the media 
content (e.g., compression format) and provide stream specific information (e.g., 
parameters for the decompressor). 

Real Networks streams can be protected by inserting a DigiBox into Header 1301 
and encrypting the data packets contained in Contoit 1302. The altered format is 
illustrated in FIG.14. which shows Header 1401, including Media Properties Headers 1402 
and 1403, which in turn contain DigiBoxes 1404 and 1405, respectively. The format also 
includes encrypted Content 1406 and Index 1407. 

In one embodiment, the declared type of the data is changed from the standard Real 
NetworiK fonnat to a new type (e.g., RNWK_Protected.) The old type is then saved. 
Changing the type forces the Real Networics player to load a 'Trust Plugin," since this 
Plugin is registered as the only decoder module that can process streams of type "RNWK- 
Protected." The Trust Plugin opens the DigiBox, gets approval from the user, if it is 
needed, determines the original content type, loads a decoder plugin for the original 
content, and then decrypts and/or validates the content, passing it to the content decoder 
plugm to be decompressed and presented to the user. 
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In one embodiment, the specific alterations made to the Real Networks file format 
are the following: 

• Increase the preroU time to force larger buffers on playback. In a current embodiment, 
an increase of 3 seconds is used. Larger buffers are needed because of the extra steps 
needed to decrypt the content. 

• Modify each stream-specific header by changing the mime type to "RNWK-Protected", 
saving the old mime type in the decoder specific information and adding a content 
identifier and DigiBox to the decoder specific information. The DigiBpx contains the 
key. initialization vector (IV). version information, and watermarking instructions. The 
key, IV and content identifier are generated automatically, or can be provided as 
command-line parameters. The same key, IV and content identifier are used for every 
stream. 

• Content packets are selectively encrypted. In one embodiment, content packets whose 
start time in milliseconds is in the first half-second of each 5 seconds (i.e., starttime % 
5000 < 500) are encrypted. This encrypts approximately one-tenth of the content 
reducing encryption and decryption costs, and damages the content, sufficiently to 
prevent resale. The encryption algorithm can be DES using output-feedback mode or 
any similar algorithm. The initiaUzation vector is computed for each packet by xoring 
the stream's IV with the packet's start time in milliseconds. Some information unique 
to the stream should also be xored into die IV. In one embodiment, the same IV is used 
for multiple packets whenever two or more streams have packets with the same start 
time. This usually happens for the first packet in each stream since they usually have 
start time 0. Other than the first packet, it is rare to have two packets have the same 
start time. 

In one embodiment, these changes to the Real Networks file fomiat are 
accomplished as is shown in FIG. 15. As is illustrated. RMFF file 1501 is formatted in the 
standard Real Networks RMFF format This file is passed to Packager 1502. Also passed 
to Packager 1502 is Rights File 1503. Packager 1503 generates Protected RMFF File 
1504, which includes various alterations as described above and as listed in FIG. 15, 
including the incorporation of one or more DigiBoxes in the header, encryption of the 
content, modification of the mime type, etc. 

In one embodiment, the trust plugin described above is illustrated in nCs. 16 and 
17. FIG. 16 illustrates the standard Real Networks aichitecture. File 1601 (e.g., a 
streaming audio file in Real Networks format) is provided to Real Networks G2 Client 
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Core 1602. File 1601 may be provided to RealNetworks G2 Client Core 1602 from Server 
1603, or through Direct Connection 1604. 

Upon receipt of File 1601 , Real Networks G2 Client Core 1602 accesses a 
rendering plugin appropriate to File 1601, based on information which is obtained from the 
header associated with File 1601. Rendering Plugins 1605 and 1606 are shown. If File 
1601 is of a type which cannot be rendered by either Rendering Plugin 1605 or Rendering 
Plugin 1606, Real Networks G2 Client Core 1602 may attempt to access an appropriate 
plugin, e.g., by asking for the user's assistance or by accessing a site associated with the 
particular file type. 

Rendering Plug-In 1605 or 1606 processes File 1601 in a conventional manner. 
This processing most likely includes decompression of File 1601, and may include other 
types of processing useful for rendering the content. Once this processing is complete 
(keeping in mind that the content is streamed, so that processing may be occulting on one 
set of packets at the same time that another set of packets is being rendered). File 1601 is 
passed back to Real Networks G2 Client Core 1602, which then passes the information to 
Rendering Device 1607. Rendering Device 1607 may, for example, be a set of stereo 
speakers, a television receiver, etc. 

FIG. 17 illustrates the manner in which a trust plugin operates within the overall 
Real Networks architecture. Much of the architecture illustrated in FIG. 17 is the same as 
that illustrated in FIG. 16. Thus, File 1701 is provided to Real Networks G2 Client Core 
1702 through Server 1703 or through Direct Connection 1704. The file is processed by 
Real Networks G2 Client Core 1702, using plugins, including Rendering Plugins 1705 and 
1706, and is then passed to Rendering Device 1707. 

FIG. 17 differs from FIG. 16 in its incorporation of Trust Plugins 1708 and 1709, 
and IRP 1710. When initially registered with Real Networks G2 Client Core 1702, Trust 
Plugins 1708 and 1709 inform Real Networks G2 Client Core 1 702 that they can process 
content of type RNWK-Protected. Whenever Real Networks G2 Client Core 1702 
encounters a stream of this type, it is then enabled to create an instance of the trust plugm 
to process the stream, e.g.. Trust Plugin 1708. It then passes the stream to the trust plugin. 

TTie stream passed to Trust Plugin 1708 may be in the format shown in FIG. 14. In 
such a case. Trust Plugin 1708 extracts DigiBox 1404 torn Media Properties Header 1402. 
It also extracts the content id and original mime type from Media Properties Header 1402. 
The Trust Plugin first checks to see if any other stream with the same content identifier has 
been opened. If so, then DigiBox 1404 is not processed fiirther. Instead, the key and IV 
from the box for this other stream are used. This avoids the time cost of opening a second 
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box. Also, this ensures that a user is only asked to pay once even if there are multiple 
protected streams. By sharing content ids. keys, and IVs. several files can be played with 
the user only paying once. This is useful when SMIL is used to play several RMFF files as 
a single presentation. 

In an alternate and possibly more secure embodiment, this check is not performed 

and the key and IV fiom die cunent DigiBox are used even if another stream with the 
content identifier has akeady beoi opened. 

If no other stream has been identified with the same content identifier. Trust Plugin 
1708 passes DigiBox 1404 to IRP 1710. IRP 1710 may be a software pn,cess running on 
the same computer as Real Networks G2 Client Core and Trust Plugin 1708. IRP 1710 
may run in a protected enviromnent or may incorporate tamper resistance techniques 
designed to render IRP 1710 resistant to attack. 

IRP 1708 may process DigiBox 1404 and extract a ciyptographic key and an IV, 
which may dien be passed to Tmst Plugin 1708. Tmst Plugin 1708 may then use this ' 
information to decrypt Encrypted Contents 1406. 

Trust Plugin 1708 uses the original mime type information extracted fiom Media 
Properties Header 1402 to create an instance of the rendering plugin to be used for the 
content (e.g.. Rendering Plugin 1705). Once this is done. Trust Plugin 1708 behaves Uke 
an ordinary rendering plugin to the Real Networks G2 Client Core 1702, in that Real 
Networks G2 Client Core 1702 passes streamed information to Trust Plugin 1708, which 
decrypts that information and passes it to Rendering Plugin 1705. From the perspective of 
Real Networks G2 Client Core 1702, Tmst Plugin 1708 constitutes the qipropriate 
rendering pluin, and the core is not aware that the information is being passed by Trust 
Plugin 1708 to a second plugin (e.g.. Rendering Plugin 1705). 

Similarly, fiom the point of view of Rendering Plugin 1705. Trust Plugin 1708 
behaves like Real Networks G2 Client Core 1702. Thus although Rendering Plugin 1705 
receives decrypted stream information fiom Trust Plugin 1708, Rendering Plugin 1705 
operates exactly as if the information had been received direcUy from Real Networks G2 
Client Core 1702. In this manner, content formatted for Rendering Plugin 1705 may 
instead be first processed by Tnist Plugin 1708. without requiring any alteration to Real 
Networks G2 Client Core 1702 or Rendering Plugin 1705. 

Trust Plugin 1708 may also perform other processing that may be helpfiil for 
security purposes. For example. Trust Plugin 1708 may watermark the decrypted file prior 
to passing it to Rendering Plugin 1705. keeping in mind that the watemiaric algorithm must 
be such diat it will survive decompression of the file by Rendering Plugin 1705. 
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MP3 Embodiment 

The techniques described above can also be applied to MP3 streaming content 
The MP-3 specification does not define a standard file format, but does define a bit 
sMchism^^^^^^ 

1805. Tk. dots between Frame 1804 and 1805 symbolize the fact that Content 1802 may 

mclude a large nmnber of fiames. 

and 180r' ^ 

Many MP3 players support a small trailer defined by the ID3 VI specification 
shown asTrailer 1809. This isal28 byte trailer for carrying fields like artist, title an^ 

year.shown as Fields 1810. 1811and 1812. -me ID3 VI trailer isignoredbyplayer. not 
designed to read such trailers, since it does not appear to be valid MP3 data. 

FIG. 19 shows one embodiment of protection applied to the MP3 format ITus 
protected foimat constitutes File 1908 and includes the following items- 

• U°»^'^»«^MP3Contentl912.Thisisthefirstinformationencounteredbya 
player, and will be rendered by any standard MP3 player. It can include a message to the 
user indicating that the content is protected and providing instructions as to how the 
content can be accessed (e.g.. a URL for a trust plugin. instructions on payment 
mechanisms, etc.) Unencrypted MP3 Content 1912 may include a 'teaser." consisting of 
an mrtial portion of the content (e.g., 30 seconds), which is rendered at no cost, thereby 
allowmg a user to sample the content prior to making a decision to purchase it. 

• Encrypted MP-3 Content 1901. which may include thousands of MP-3 frames 

Inoneembodiment.thefirsteightftamesoutofevery32framesareencrypted. TT^us one- 
quareter of the frames are rendered unuseable unless a player is able to decrypt them 'in 
practice, this may render the content un-sellable or unuseable. without imposmg excessive 
encryption or decryption costs. To further reduce encryption and decryption costs, only 32 
bytes m each frame are encrypted. In a current embodiment, these are the first 32 bytes 
after the header and CRC information. In a different embodiment, a different 32 bytes may 
be encrypted in every frame. In a cun^it embodiment, the content is encrypted with the 
DES usmg algorithm output-feedback mode. THe initial IV for the file is randomly 
generated and then xored with the frame number to generate a unique IV for each frame 

Many alternate embodiments may exist, including encrypting more or less 
information, and using diflferent encryption algorithms. 

• ID3 VI Trailer 1902. including 128 bytes. 
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• Content ID 1903. including 16 bytes. This is used by the player application to 
avoid opening DigiBoxes which it has already opened. 

• DigiBox 1904. which may comprise approximately 1 8K bytes. It includes Key 
1909. IV 1910 and Watermarking Instructions 1911. Watermarking Instructions 1911 may 
be used in a process of watermarking the associated content 

• Address 1905. which contains the address in the file of Content ID 1903 and 
consists of 4 bytes. 

• Trust ID 1906. which identifies this trusted MP-3 file and consists of 16 bytes. 

• ID3 VI Trailer 1 907. which is a copy of Trailer 1902. 

A conventional MP3 player encountering File 1908 would be unable to render 
Content 1901. since at least a portion ofthat content is encrypted. Such a player would 
most likely read through to Trailer 1902 and cease processing at that point Aconventional 
player looking for the ID3 trailer information will seek to the end and find it. 

FIG. 20 illustrates one embodiment of an MP3 player designed to process and 
render protected content This figure shows MP3 Player 2001. which includes Buffer 2006 
and Decompressor 2007. and renders content to Rendering Device 2008. In one 
embodiment, this is a modified version of a player distributed by Sonique. 

Player 2001 obtains Protected MP3 File 2002 through any standard interface. 
Protected MP3 File 2002 may have the format illustrated in FIG. 19. 

When Player 2001 is asked to play Protected MP3 File 2002, Player 2001 first calls 
Trust Plug-In 2003. which includes Approval Function 2009 and Decrypt Function 2005. 
Trust Plugin 2003 calls Approval Function 2009 to determme if Protected MP3 File 2002 
is protected and whether authorization exists to play the file. Approval Function 2009 is 
first given a pointer to Protected MP3 File 2002. It then checks Protected MP3 File 2002 
for the presence of Trust ID 1906. If Trust ID 1906 is not found. Approval Function 2009 
returns an indicator that the file is not protected. Player 2001 then proceeds to render the 
file as a normal MP3 file. 

If Trust ID 1906 is found. Approval Function 2009 checks Content ID 1903 to see 
if it matches the Content ID of a file that has already been opened. 

If Protected MP3 File 2002 has not been previously opened. DigiBox 1904 is 
retrieved by Approval Function 2009, and is passed to mp 2004. which may include 
software nmning in a protected enviromnent, or incorporating tamper resistance. IRP 2004 
attempts to open DigiBox 1904 in compHance with the nUes associated with that DigiBox. 
One such rule may require, for example, that the user mdicate assent to pay for use of the 
content If DigiBox 1904 camiot be opened (e.g.. the user refiises to pay) a value is 
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retumed to Approval Function 2009 indicating that the file is protected and may not be 
played. 

If DigiBox 1904 is opened in compUance with applicable rules, the key and IV are 
retrieved and passed to Decrypt Function 2005. The key and IV are stored with the content 
Id for later re-use and Decrypt Function 2005 is initialized. This may improve overall 
system perfomiance, since it reduces the number of times a DigiBox must be opened. Each 
such action may introduce significant latency. 

On the other hand, storing this information in unprotected memory may reduce 
overall system security. Security may be enhanced either by not storing this information 
(thereby requiring that each DigiBox be opened, even if the corresponding file has aheady 
been opened through another DigiBox). or by storing this infomtation in a protected form 
or b a secure location. 

The stored key. IV and content id are referenced when Approval Function 2009 first 
checks Content ID 1903 to detennine if it matches the Content ID of an already opened 
file. If the new Content ID matches a stored Content ID. Decrypt Function 2005 is 
reinitialized using the stored key and IV conesponding to the matching content id an. 
value indicating that this is a protected file for which play is authorized is returned to 
Approval Function 2009. 

Once Protected MP3 File 2002 has been opened, each time Player 2001 needs a 
packet. Player 2001 reads it into Buffer 2006. strips off the header and CRC and passes the 
remaining data and a fi^e number to Decrypt Function 2005, which decrypts the fiame if 
necessary, and returns it to Player 2001. 

In a current embodiment, although audio content is encrypted, headers or trailers 
are not encrypted. TWs allows the Player 2001 to process infomiation in headers or trailers 
without intervention from Approval Function 2009 or Decrypt Function 2005. This allows 
Player 200 1 to place information such as playing time, artist and title into a playlist display, 
and initialize Decompressor 2007. without any action required fix>m Trust Plugin 2003. 

Commerce Appliance Embodiment 

This section will describe an embodiment, comprising a Commeix^e Appliance 
architecture designed to allow persistent control of digital works in consumer electronics 
devices. Although this is described as a separate embodiment, it should be understood that 
the features of this embodiment may be combined with, or supplant, the features of any of 
the embodiments provided elsewhere m this descriptioa 

In one embodiment, this section will describe modifications to the MPEG-4 
standard designed to support the association of persistent rules and controls witii MPEG-4 
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content, as well as elements necessary for a Commerce Appliance to use such content. 
This is intended, however, merely as an example. 

In one embodiment, shown in FIG. 23, each Commerce Appliance 2301 includes a 
CMPS ("Content Management and Protection System") 2302. Each CMPS is responsible 
for governing the use of controlled content, including decrypting the content and ensuring 
that the content is only used as permitted by associated rules. 

Each governed digital work is associated with one or more CMPOs (Content 
Management Protection Object), e.g., CMPOs 2303. Each CMPO may specify rules 
governing the use of the digital work, and may include keys used to decrypt the work. 

CMPOs may be organized in an hierarchical fashion. In one embodiment, a content 
aggregator (e.g.. a cable channel, a web site, etc.) may specify a Channel CMPO 
("CCMPO") used to associate certain global rules with all content present on that chamiel. 
Each independent work may in turn have an associated Master CMPO ("MCMPO") used to 
associate rules applicable to the work as a whole. Each object (or Elementary Stream, in 
MPEG-4) may have associated with it a CMPO containing rules governing the particiilar 
object. 

In one exemplary application. Commerce Appliance 2301 may be an MPEG-4 
player containing CMPS 2302. Upon receipt of a user command to play a particular work, 
CMPS 2302 may download a MCMPO associated with the work and obtain rules, which ' 
may include conditions required for decryption and viewing of the work. If the rules are 
satisfied. CMPS 2302 may use keys ftom the MCMPO to decrypt any Elementary Streams 
("ES"), and may pass the decrypted ESs into the buffers. Composition and rendering of the 
MPEG-4 work may thereafter proceeds according to the MPEG-4 standard, except that any 
storage location or bus which may contain the work in the clear must be secure, and CMPS 
2302 may have the abUity to govern downstream processing, as weU as to obtain 
information regarding which AVOs were actually released for viewing. 

In a variation, the process of obtaining and governing the work may include 
downloading a CCMPO which applies rules governing this and other works. If rules 
contained in the CCMPO are satisfied. CMPS 2302 may obtain a key used to decrypt the 
MCMPO associated with the particular work to be viewed. 

In another variation, a CMPO may be associated with each ES. In this variation, 
the MCMPO siq)plies one or more keys for decryption of each CMPO, and each CMPO 
may in turn supply a key for decryption of the associated ES. 

Commerce ^liance 2301 is a content-rendering device which mcludes the 
capability of supporting distributed, peer management of content related rights by securely 
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applying rules and controls to govern the use of content. Commerce AppUance 2301 may 
include genecal-pun^ose functions devoted to acquisition and managed rendering of content 
(e.g.. a DVD (and/or any other optical disk format) player is able to play a DVD (and/or 
any other optical disk format) disk and output content to a television.) Commerce 
Appliance 2301 may make use of any of the means for protecting and using digital content 
on high capacity optical disk, in one non-limiting example, a DVD disk, as described in the 
aforementioned Shear patent application. 

Commerce AppUance 2301 also includes special-purpose functions relating to other 
management and protection of content functions. These special-purpose functions may be 
supported by one or more embedded or otherwise included CMPS 2302 in the fomi of a 
smgle CMPS or a cooperative CMPS arrangement, and may include a user interfece (e g 
User Interface 2304) designed to display control-related infomiation to the user and/or to' 
receive conHol-related information and directions from the user. Commerce Appliance 
2301 may also be designed so tiiat it is networkable with other Commerce Appliances (e g 
a set-top box comiected to a DVD player and a digital television) and/or with other devices' 
such as a computer arrangement, which may also include one or more CMPSs. 

An important fonn of Commerce Appliance specifically anticipates secure couphng 
on a penodic or continual fashion with a computer managed docking enviromnent (e g a 
standalone computer or otiier computer managed device which itself may be a Commerce 
Appliance) where the one or more CMPSs of tiie Commerce Appliance interoperate with 
the docking enviromnent to form a single user arrangement whose perfomiance of certain 
functions and/or certain content usage events is enabled by such inter-operation through at 
least m part, cooperation between CMPSs and content usage management information o'f 
the Commerce Appliance and the trust enviromnent capabiUties of the docking 
enviromnent. (e.g.. further one or more CMPSs and content usage management 
information, such as, for example, information provided by use of CI). 

An exemplary Commerce AppUance may be designed to comply with the emerging 
MPEG-4 standard for the formatting, multiplexing, transmission, compositing, and 
rendermg of video and other types of information. 

Commerce Appliance 2301 may be any computing device, one non-limiting 
example of which is a Personal Computer (PQ that includes MPEG-4 software (and/or 
hardware) for rendering content In accordance with the present invention, the PC may 
also use one or more CMPSs as described herem. 

The commerce appUance function is not restricted to streamed chamiel content but 
may mclude various browser-type applications consisting of aggregated composite content 



SUBSTITUTE SHEET (RULE 26) 



wo 99/48296 

PCTAJS99/0S734 

-38- 

such as stiU inu^gery. text, synthetic and natural video and audio and functional content 
such as applets, animation models and so on. these devices include browsers, set-top 
boxes, etc. ^ 

Content Management and Protection System (CMPS) 
Each commerce appUance includes one or more CMPS (e.g.. CMPS 2302) ITie 
CMPS IS responsible for invocation and appUcation of rules and controls, including the use 
of rules and controls to govern the manner in which controlled content is used. 
Particular functions of CMPS 2302 include the following: 

(a) Identification and interpretation of rules. 

CMPS 2302 must determine which rules are to be applied, and must determine how 
those rules are to be interpreted in light of existing state information. In one embodiment, 
this requires that CMPS 2302 obtain and decrypt one or more CMPOs 2303 associated 
with a work. 

(b) Identification of content associated with particular rules. 

CMPS 2302 must determine which content is governed by particular one or more 
niles. This may be accomplished by obtaining information from one or more CMPOs 2303 
and/or other Q. In one embodiment, a CCMPO may identify a set of works, a MCMPO 
may Identify a particular work and a CMPO may identify a particular ES or Audio Visual 
Object ("AVO"). 

(c) Decryption of content as allowed by the rules. 

CMPS 2302 may be designed so that all content is routed through CMPS 2302 for 
decryption, prior to reinsertion into the data flow required by the relevant standard. In the 
case of MPEG-4, for example, the output from Demux 2305 may be fed into CMPS 2302 
CMPS 2302 may then decrypt the content and, if relevant mles and controls are satisfied, 
feed the content into the MPEG-4 buffers. From that point, the data flow associated with 
the content may be as described by MPEG-4. 

(d) Control of content based on rules. 

CMPS 2302 may be used to control usage of content after the initial decryption, for 

example, through the use of secure event management as described in the incorporated 
Gmter '333 patent appHcation. In the case of MPEG-4 sy^ this may require that 
CMPS 2302 exercise control over hardware and/or software which performs the following 
fimctions: demuxing (perfomied by Demux 2305). decompression/bufFering/decode into 
AVOs (performed by Scene Descriptor Graph 2306, AVO Decode 2307 and Object 
Descnptors 2308), scene rendering (performed in Composite and Render 2309) 



SUBSTITUTE SHEET (RULE 26) 



wo 99/48296 „^ 

PCT/US99/0S734 

-39- 

CMPS 2302 may also be used to control use and consequences according to: (1) 
generational copy protection rules such as the CGMS and/or SGMS standards; (2) various 
Conditional Access control methods, such as those proposed and/or implemented by NDS 
as described in MPEG-4 document M2959. DAVIC "Copyright Control Framework- 
document, and in other publications; (3) a Rights Management Language, such as those 
proposed in the Ginter '333 patent application and/or as described by U.S. Patent No. 
5,638. 443 to Stefik. et al.; (4) use policies described in accordance with AT&T's Policy 
Maker, as described by Blaze, Feigenbaum. and Lacy; (5) the CCI layer bits for IEEE 1 394 
serial bus transmission as specified by the DTDG subgroup of the DVD Copy Protection 
Technical Woridng Group and/or as implemented by the Hitachi. Intel, Matsushita. Sony 
and Toshiba proposed standard (hereafter "the five company proposal"); (6) controls 
transmitted using any secure container technology such as, for example. fflM Cryptolope; 
(7) any other means for specifying use rules and consequences, 
(e) Monitoring use of content. 

CMPS 2302 may be used to monitor content to: (i) ensure that rules are being 
compUed with; (ii) ensure that no attempts are being made to tamper with tiie system or 
protected content; and (ui) record information used by rules, including usage information 
needed for payment purposes. 

(0 Updating user budgets. 

CMPS 2302 may be used to update user or other budgets to reflect usage. 

(g) Exhaust information. 

CMPS 2302 may be used to output payment and usage infonnation ("exhaust 
mformation") to external processes, including one or more Commerce Utility Systems. 

(h) Hardware identification and configuratioa 

(i) Obtaining new. additional, and/or augmented rules from an external 
process, one non-lhniting example of which is a Rights and Permission Clearinghouse as 
described in tiie incorporated Shear patent application. 

0) Receiving keys, digital credentials, such as certificates, and/or 
administrative information, fiom certifying autiiorities, deployment managers, 
clearinghouses, and/or oUier tiusted infrastructure services. 

(k) Securely sending and/or receiving user and/or sqjpUance profiling and/or 
attribute information. 

0) Securely identifying a user or a member of a class of users who requests 
content and/or CMPO and/or CMPS usage. 
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(m) Securely certifymg or otherwise guaranteeing the authenticity of 
application code, for example certifying within CMPO 2301 and/or CMPS 2302 that 
application code containing rules and/or other appUcation information, such as information 
wntten in Java code for conditional execution within a Commerce Appliance, and/or that 
executes at least in part outside of CMPO 2301 and/or CMPS 2302. has not been altered 
and/or has been deUvered by a guaranteed (e.g.. trusted) party. 

(n) Securely processing independently deUvered CI. such as described in the 
mcorporated Ginter '333 patent appUcation. to perform content usage control that protects 
the nghts of plural, independent parties in a commerce value chain. 

(0) Securely performing watermarking (including, for example fingerprinting) 
functions, for example as described in the Ginter '333 patent application and as 
mcorporated herein, for example including interpreting watennaridng infomration to 
control content usage and/or to issue an event message, wherein such event message may 
be reported back to a remote authority, such as, for example, a MCMPO rights 
clearinghouse management location. 

CMPS 2302 may be used to identify and record the current hardware configuration 
of the Commerce Appliance and any comiected devices (e.g., which loudspeakers are 
available, identification of attached monitors, including whether particular monitors have 
digital output ports, etc.) If attached devices (such as loudspeakers) also include CMPSs 
the CMPSs may be used to communicate for purposes of coordination (e.g.. a CMPS in a 
set-top box and/or loudspeaker arrangement may communicate with a CMPS in a 
downstream digital television or other display device to establish which CMPS will be 
responsible for governance or the nature of cooperative governance through a virtual rights 
process, said process optionaUy involving a rights authority server that may find, locate, 
provide, aggregate, distribute, and/or manage rights processes, such as described in the ' 
aforementioned Shear patent application, for employing plural CMPSs, for example, for a 
single user content processing and usage arrangement). 

The present invention includes arrangements comprising plural Commerce 
AppUances and/or CMPSs in one or more user locations, non-limiting examples of which 
mclude a home, apartment, loft, office, and/or vehicle, such as a car. truck, sports utility 
vehicle, boat, ship, or airplane, that may communicate among themselves at least 
occasionally and may comprise a virtual network that operates in a logicaUy cooperative 
mamier, through at least in part the use of such CMPSs. to ensure optimal commercial 
flexibiUfy and efficiency and the enforcement of rights of commerce value chain 
participants, including financial and copyright rights of providers, infrastructure rights of 
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apphance providers, societal rights of government and/or societal bodies, and privacy 
nghts of all parties, including consumers. Information related to interaction among such a 
networic of value chain participants, including content usage auditing, content usage 
consequence, and CI specification, can be securely, variably reported to parties having right 
to such information, through, at least in part, use of such CMPSs. for example, as described 
m the aforementioned Ginter 712 patent application regarding the information reporting 
functioning of VDE nodes. 

In one embodiment, shown in FIG. 24. CMPS 2401 consists of special-pu^Jose 
hardware and resident software or firmware. TTiese include the foUowing: 

(a) One or more processors or microcontrollers e.g. CPU 2402 CPU 2402 
controls the overall processing of CMPS 2401. including execution of any necessary 
software. 

(b) One or more external communications ports, e.g., Port 2403 Port 2403 
communicates with External Network 2404, which may include LANs, WANs or 
distnbuted networks such as the Internet. External communications ports may also include 
one or more IEEE 1394 serial bus interfaces. 

(c) Memory 2405. Types of memories which may be included in Memory 
2405- and examples of the information they may store - are the following: 

i. ROM 2406. ROM 2406 may include any information which is 
permanently stored in CMPS 2401. such as (1) CMPS Operating System 2407 and/or 
CMPS BIOS 2408. (2) Rule^Controls 2409 which are pennanently stored in the CMPS- 
(3) Control Primitives 2410 which may be used to build rules or controls; (4) Keys 241 1' 
associated with the CMPS. including a Public/Private Key Pair; (5) one or more 
Certificates 24 12 designed to identify CMPS 2401 and/or the device, including version 
mformation; (6) Hardware Signature Information 2413 used to check for tampering (e g a 
hashed signature reflecting the expected hardware state of the device). 

ii. RAM 2414. RAM 2414 may hold cmrent state infomiation 
needed by CMPS 2401. as well as information temporarily stored by CMPS 2401 for later 
use. Infomiation stored in RAM 2414 may include the following: (1) Software2415 
currenUy executing in CPU 2402; (2) CMPOs 2416 which are currently active; (3) Content 
Object Identification 241 7 of those content objects which are currenUy active (in an MPEG 
4 system this would constitute, for example, an identification of active AVOs)- (4) Rules 
241 8 which are currently active; (5) State Infomiation 2419 regarding the current state of 
use of content, including an identification of any higher-order organization (in an MPEG-4 
system this would constitute an identification of the scene descriptor tree and the current 
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state of composition and rendering); (6) Stored Exhaust Information 2420 relating to use 
and/or the user, designed for external transmission; (7) Updated Budget Information 242 1 • 
(8) Content 2422; (9) Active Content Class Information 2423; and (10) Active User 
Identification 2424, including identification characteristic infonnation. 

iii- NVRAM 2425 (e.g.. flash memory). This type of memory may 
hold mformation which is persistent but changeable, including at least some: (1) Budget 
Information 2426; (2) User Infomiation 2427. such as identification, credit card numbers- 
preferred clearinghouses and other Commerce UtiUty Systems; (3) User Preferences 2428 
such as preferences, profiles, and/or attiibute information; and (4) AppUance Information 
2429. such as attribution and/or state information. 

The types of information described above and stored in CMPS Memory 2405 may 
be stored in alternative of the above memory types, for example, certain budget information 
may be located in ROM. infomiation regarding specific one or more clearinghouses may be 
stored m ROM. certain active information may be moved into NVRAM, etc. 

Budget information may include stored budgets made up of, for example: 

(1) electiionic cash; 

(2) pre-authorized uses (e.g., based on a prepayment, the user has the right 
to watch 12 hours of programming). 

(3) Security budgets related to patterns reflecting abnormal and/or 
unauthorized usage, for example, as described in the incorporated Shear 
patent, wherein such budgets restiict and/or report certain cumulative 
usage conduct. 

(4) electronic credit, including credit resulting from usage events such as 
attention to promotional material and/or the playing of multiple works 
fiom one or more classes of works (e.g.. certain publisher's works) 
triggering a credit or cash refimd event and/or a discount on fiiture 
playing of one or more of such publisher's worics, such as other works 
provided by such publisher. 

User information may include the following types of information for one or more 
authorized users of the Commerce Appliance: 

(1) Name, address, telephone number, social security number or other 

identifier 

(2) Infonnation used to authenticate the user, which may include a user 
selected password and/or biometric data, such as fingerprints, retinal data. etc. 

(3) User public/private key pair 



SUBSTITUTE SHEET (RULE 26) 



wo 99/48296 



PCT/US99/05734 



•43 



(4) User attribute and/or profiling information. 

iv. Removable Memory 2430. This may include any type of 
removable memory storage device, such as smart cards, 
floppy disks or DVD disks. If the commerce appliance is 
designed to play content received on removable memory 
devices (e.g., a DVD player), that capability may be used for 
purposes of the CMPS. 
Memory 2405 may include a protected database, in which certain control, budget, 
audit, security, and/or cryptographic information is stored in secure memory, with complete 
information stored in an encrypted fashion in unsecure memory. 

(d) Enciyption/DecTyption Engine 2431. CMPS 2401 must include a 
facility for decrypting received information, including content and CMPOs and/or other. 
CMPS 2401 may also include a feciUty for encrypting information if such information is to 
be transmitted outside the secure boundaries of CMPS 2401 . This may include exhaust 
sent to clearinghouses or other external repositories; and content sent across unsecured 
buses for usage, such as content sent across IEEE 1394 Serial Bus 2432 to a computer 
central processing arrangement or to a viewing device such as a monitor, wherein a 
receiving CMPS may be employed to control such content's usage, including, for example, 
decrypting such content, as appropriate. Encryption^ecryption Engine 2431 may include' 
a Random Number Generator 2433 used for the creation of keys or key pairs that can be 
used to identify and assure the uniqueness of CMPSs and support the opening of secure 
communication chamiels between such secure content control secure encryption/decryption 
arrangements. 

(e) Secure Clock/Calendar 2434. CMPS 2401 may include Secure 
Clock/Calendar 2434 designed to provide absolute information regarding the date and time 
of day, information regarding elapsed absolute time, and/or relative timing information 
used to determine the elapsed time of operations performed by the system. Secure 
Clock/Calendar 2434 may include Battery Back Up 2435. It may fiirther include Sync 
Mechanism 2436 for synchronization with outside timing information, used to recover the 
correct time in the event of a power loss, and/or to check for tampering. 

(0 Interfece 2437 to blocks used for content rendering and display. This 
interface is used for controlling rendering and display, based on rules, and for obtaining 
feedback information, which may be used for budgeting purposes or for providing 
information to outside servers (e.g.. information on which content was actuaUy displayed, 
which choices the user invoked, etc.) hi the case of an MPEG-4 player such as is shown in 
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FIG. 23, this may include control over Commerce Appliance circuitry which handles, for 
example, buffering, the scene descriptor graph, AVO decode, object descriptors and 
composite and rendering (e.g.. Control Lines 2310. 2311 and 2312). 

Feedback Path 2313 ftom Composite and Render block 2309 may allow CMPS 
2302 to determine whether and when content has actually been released to the viewer. For 
example. Composite and Render block 2309 can issue a start event to CMPS 2302 when ar 
AVO object is released for viewing, and can issue a stop event to CMPS 2302 when the 
AVO object is no longer being viewed. 

Feedback from Composite and Render block 2309 may also be used to detect 
tampering, by allowing CMPS 2302 to match the identification of the objects actually 
released for viewing widi the identification of the objects authorized for release. Start and 
end time may also be compared with the expected elapsed time, with a mismatch possibly 
indicative of the occurrence of an unauthorized event. 

In one embodiment, the following protocol may be used for feedback data: 
start <id>, T, <instance nomberxclock timexrendering options> 

Sent if elementary stream <id> is reachable in the SD-graph at time T, but not at 

time T-1. 

end <id>, T, <instance numberxclock timexrendering options> 

T constitutes presentation time, clock time constitutes the wall clock time, including day 
and date information, and rendering options may include such information as QoS and rate 
of play (e.g., fast forward). 

Sent if elementary stream <id> is reachable in the SD-graph at time T-l but not at 
time T. A SD-graph stream is reachable if; during traversal of the SD-graph for display 
update, the renderer encounters a node that the SD-graph update stream <id> created or 
modified. This impUes that all nodes in the tree need an update history list. TTiis Ust need 
not be as large as the number of streams. Further, it can be labeled to indicate if the CMPS 
wUl be watching for stream, if not labeled it wiU not record them. An AV elementary 
stream is reachable if the stream's content was rendered. 

For SD-graph update streams, the object instance number is ignored. For AV 
streams, the instance number can be used to disambiguate the case where the display shows 
two or more instances of the same data stream simultaneously. Instance numbers do not 
have to count up. In this case, they are simply a unique id that allows the CMPS to match a 
start evait with an end event 

In a second embodiment. CMPS 2302 may include some special purpose hardware 
in combination with general purpose hardware which is also used for other functions of the 
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device. In this embodiment, care must be taken to ensure that commercially trusted CMPS 
functions are performed in a secure and tamper-resistant manner, despite the use of general 
purpose hardware. Each of the elements recited above may include dedicated CMPS 
functions and general purpose device functions: 

(a) CPU/microcontroller. This may include one or more devices. If more 
than one device is included (e.g., a CPU and a DSP. a math coprocessor or a commerce 
coprocessor), these devices may be included within the same package, which may be 
rendered tamper-resistant, or the devices may communicate on a secure bus. TTie CPU may 
include two modes: a secure CMPS mode, and an unsecure general purpose mode. The 
secure CMPS mode may allow addressing of secure memory locations unavailable to the 
processor in general purpose mode. This may be accomplished, for example, by circuitry 
which remaps some of the available memory space, so that, in mrsecure mode, the CPU 
cannot address secure memory locations. 

(b) External communications ports. If the device, for example, a Commerce 
Appliance, is capable of receiving content or other information through a communications 
port (e.g., a cable comiection, an Internet comiection), this communications port can be 
used for CMPS purposes. In such a case. CMPS accesses to the external communications 
port is preferably designed to avoid or minimize interference with the use of such port for 
receipt of content. 

(c) Memory. In some applications and embodiments, it is possible to 
operate a Commerce AppUance without NVRAM, wherein information that may be needed 
for CMPS operation that would employ NVRAM would be loaded into RAM, as required. 
ROM. RAM and NVRAM may be shared between CMPS uses and general uses. This can 
be accomplished in any of the following ways, or in a combination of these ways: (1) 
Some memory space may be rendered off-limits to general purpose uses, for example by 
rem^ping; (2) the entirety of the memory may be rendered secure, so that even portions of 
the memory being used for non-secure purposes camiot be observed or changed except in a 
secure and authorized mamier; (3) CMPS information may be stored in an encrypted 
fashion, though this requires at least some RAM to be secure, since the CMPS will require 
direct access to unencrypted information stored m RAM. 

(d) Encryption/decryption engine. Encryption and decryption functions, 
including key generation, may be handled by special purpose software rmrning on a general 
purpose processor arrangement, particularly, for example, a floating point processor or 
DSP arrangement. That processor arrangement may also be used for purposes of 
decompressing and displaying content and/or for handling watermarking/fingerprinting 
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insertion and/or reading. Alternatively, the device may include native encryption and 
decryption functions. For example, various emerging standards may require at least some 
degree of encryption and decryption of content designed to be passed across unsecure buses 
within and among devices such as DVD players, such as the "five company proposal" and 
5 other IEEE 1394 related initiatives. Circuitiy designed to perform such encryption and 

decryption may also be usable for CMPS applications. 

(e) Secure clock/calendar. The underlying device may already require at 
least some clock information. MPEG-4, for example, requires the use of clock information 
for synchronization of Elemaitary Streams. A secure CMPS clock can also be used for 
10 such purposes. 

In a third embodiment, CMPS 2302 can be primarily software designed to run on a 
general purpose device which may include certain minimal security-related features. In 
such a case, CMPS 2302 may be received in the same chamiel as the content, or in a side- 
band channel. An I-CMPO and/or other CI may specify a particular type of CMPS, which 

1 5 Commerce AppUance 2301 must either have or acquire (e.g., download from a location 

specified by the I-CMPO), or CMPS 2302 may be included, for example, with an I-CMPO. 

A software CMPS runs on the CPU of the Commerce AppUance. This approach 
may be inherently less secure than the use of dedicated hardware. If the Commerce 
Appliance includes secure hardware, the software CMPS may constitute a downloadable 

20 OS and/or BIOS which customizes the hardware for a particular type of commerce 

application. 

In one embodiment, a software CMPS may make use of one or more software 
tamper resistance means that can materially "harden" software. These means include 
software obfiiscation techniques that use algorithmic means to make it very difficult to 
reverse engineer some or all of a CMPS, and fiirther make it difficult to generalize from a 
reverse engineering of a given one or more CMPS. Such obfiiscation is preferably 
independent of source code and object code can be different for different CMPSs and 
different platfomis, adding fiirther complexity and separation of roles. Such obfiiscation 
can be employed "independently" to both CI. such as an CMPO, as well as to some or all 
30 of the CMPS itself; thus obscuring both the processing enviromnent and executable code 

for a process. The approach is also applicable for integrated software and hardware 
implementation CMPS implementations described above. Other tamper resistance means 
can also be employed, including using "hiding places" for storing certain state information 
in obscure and unexpected locations, such as locations m NV memory used for other 
purposes, and data hiding techniques such as watermarking/fingeiprinting. 
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Association of CMPS With a Commerce Appliance 

A CMPS may be permanently attached to a particular device, or may be partially or 
fully removable. A removable CMPS may mclude software which is securely loaded into a 
Commerce Appliance, and/or removable hardware. A removable CMPS may be 
personalized to one or more particular users, including user keys, budget information, 
preferences, etc., thereby allowing different users to use the same Commerce Appliarice 
without commingling budgets and/or other rights, etc. 

A CMPS may be designed for operation with certain types of content and/or for 
operation with certain types of business models. A Commerce Appliance may include 
more than one type of CMPS. For example, a Commerce Appliance designed to accept 
and display content pursuant to different standards may include one CMPS for each type of 
fonnat. In addition, a Commerce Appliance may include a CMPS provided by a particular 
provider, designed to preferentially display certain types of content and to preferentially 
bill for such content through a particular channel (e.g., billing to one or more particular 
credit cards and/or using a particular one or more clearinghouses). 
Source of Rules 

The CMPS must recognize those rules which are to be applied to particular content 
Such rules may be received by the CMPS fiom a variety of sources, depending on the 
particular embodiment used: 

(a) CMPO. The rules may be included within a CMPO (e.g., CMPO 2303) 
and/or oAer CI. Tlie CMPO and/or other CI may be incorporated within a content object 
or stream (as, e.g., a header on an MPEG-4 ES), and/or may be contained within a 
dedicated content object or stream encoded and received as per the underlying standard 
(e.g., an MPEG-4 CMPO ES). and/or may be received outside the normal content stream, 
in which event it may not be encoded as per the underlying standard (e.g., a CMPS 
received as an encrypted object through a sideband channel). 

(b) CMPS. Rules may be permanently and/or persistenUy stored within a 
CMPS, e.g., Rules 2409. A CMPS may include default rules designed to handle certain 
situations, for example, where no CMPO and/or other necessary CI is received (e.g., 
content encoded under an earUer version of the standani which did not incorporate CMPOs. 
including MPEG-4 version 1). Complete rules which are stored withm the CMPS may be ' 
directly or indirectly invoked by a CMPO and/or other CI. This may occur through the CI 
identifying particular rules through a pointer, and/or it may occur through the CI 
identifying itself and the general class of control it requires, with the CMPS then applying 

35 particular rules specific to that CMPS. 
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Rule "primitives" may also be stored within the CMPS (e.g., Control Primitives 
2410). The CMPO and/or other CI may invoke these primitives by including a sequence of 
macro-type commands, each of which triggers a sequence of CMPS primitives. 

(c) User. The user may be given the ability to create rules relating to the 
particular user's preferences. Such rules will generally be allowed to further restrict the use 
of content, but not to expand the use of content beyond that which would otherwise be 
allowed. Examples include: (a) rules designed to require that certain types of content 
(e.g., adult movies) only be accessible after entry of a password and/or only to certain 
CMPS users (e.g. adults, not children, as, for example, specified by parents and/or a 
societal body such as a government agency); (b) rules designed to require that only 
particular users be allowed to invoke operations requiring payment beyond a certain limit 
and/or aggregate payment over a certain amount. 

The user may be allowed to create templates of rules such as described in the 
aforementioned Ginter '333 patent application (and incorporated herein). In addition, a 
CMPS arrangement, and/or a particular CMPO and/or other CI, may restrict the rules' the 
user is allowed to specify. For example, a CI may specify that a user can copy a woric, but 
cannot add rules to the work restricting the ability of a recipient to make additional copies 
(or to be able to view, but only after a payment to the first user). User supplied one or 
more rules may govern the use of - including privacy restrictions related to - payment, 
audit, profiling, preference, and/or any other kind of information (e.g., information result as 
a consequence of the use of a CMPS arrangement, including, for example, use of secured 
content). Such user supplied one or more mles can be associated with the user and/or one 
or more Commerce Appliances in a user arrangement, whether or not the infonnation is 
aggregated according to one or more criteria, and whether or not user and/or appliance 
identification information is removed during aggregation and/or subsequent reporting, 
distribution, or any other kind of use. 

The ability to allow the user to specify rules allows the CMPS to subsume (and 
thereby replace) V-chips, since a parent can use content rating information to specify 
precisely what types of information each viewer wiU be allowed to watch (e.g., violent 
content can only be displayed after entry of a certain password and/or other identifier, 
including, for example, insertion of a removable hardware card (smart or rights card) 
possessed by a user). 

(d) External network source. The rules may be stored on an external server. 
Rules may be addressed and downloaded by the CMPS if necessary (e.g., either the CMPO 
and/or other CI and/or the CMPS contains a pointer to certain rules location(s). such as one 
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or more URLs). In addition, content providers and/or clearinghouses may broadcast rules 
designed for general appHcability. For example, a content provider might broadcast a set 
of rules providing a discount to any user participating in a promotional event (e.g., by 
providing certain user infomiation). Such mles could be received by all connected devices, 
could be received by certain devices identified as of interest by the content provider (e.g., ' 
all recent viewers of a particular program, as identified by exhaust information provided by 
the CMPS to a clearinghouse and/or aU members having certain identity characteristics 
such as being members of one or more classes) and/or could be posted in central locations. 
Example Embodiment 

In one embodiment, a set of MPEG-4 Elementary Streams may make up a wo±. 
The Elementary Streams may be encrypted and multiplexed together to form an Aggregate 
Stream. One or more CMPOs may be present in such stream, or may otherwise be 
associated with the stream. Options are as follows: 

1 . Content may be streamed or may be received as static data structures. 

2. A Work may be made up of a single stream or data structure, or of many 
separately addressable streams or data structures, each of which may constitute an Object. 

3. If a Work is made up of separately addressable streams or data structures, those 
streams or data structures may be multiplexed together into an Aggregate Stream, or may 
be received separately. 

4. If streams or data stmctures are multiplexed together into an Aggregate Stream, 
the streams or data structures may be encrypted prior to such multiplexing. The Aggregate 
Stream itself may be encrypted, whether or not the underlying streams or data stmctures are 
encrypted. The following possibilities therefore exist: (a) individual streams/data 
structures are unencrypted (in the clear), the Aggregate Stream is unencrypted; (b) 
individual streams/data structures are unencrypted prior to multiplexing, the Aggregate 
Stream is encrypted following multiplexing; (c) individual streams/data structures are 
encrypted prior to multiplexing, the Aggregate Stream is not encrypted following 
multiplexing; or (d) individual streams/data stmctures are encrypted prior to multiplexing, 
the Aggregate Stream is encrypted following multiplexing. 

5. A CMPO may be associated with a channel (CCMPO), a work (MCMPO) or an 
individual Object (CMPO). 

6. A CMPO may be received prior to the controlled data, may be received 
contemporaneously with the data, or may be received after the data (in which event use of 
the data must wait until the CMPO has been received). 

7. A CMPO may be received as part of an Aggregate Stream or separately. 
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8. Ifa CMPO is received as part of the Aggregate Stream, it may be multiplexed 
together with the individual streams or data structures, or may constitute a separate stream 
or data structure. 

9. If a CMPO is multiplexed within the Aggregate Stream, it may be encrypted or 
nonencrypted. If encrypted, it may be encrypted prior to multiplexing, and/or encrypted 
after multiplexing, if the entire Aggregate Stream is encrypted. 

10. Ifa CMPO is received as part of the Aggregate Stream, it may be (a) a part of 
the stream or data structure which holds the content (e.g., a header); (b) a separate stream 
or data structure encoded pursuant to the same format as the streams or data structures 
which hold the content (e.g.. an MPEG-4 ES) or (c) a separate stream or data structure 
encoded under a different format designed for CMPOs. 

1 1 . If a CMPO is a part of the stream or data structure which holds the content, it 
may be (a) a header which is received once and then persistently maintained for control of 
the content; (b) a header which is received at regular intervals within the stream or data 

1 5 structure; or (c) data distributed throughout the stream or data structure. 

These various scenarios give rise to different requirements for demultiplexing and 
decryption of the CMPOs. FIG. 25 illustrates the foUowing embodiment: 

1. Aggregate Stream 2501 is made up of multiplexed ESs (e.g., ES 2502 and 2503). 
A combination of such ESs makes up a single woric Aggregate Stream 2501 is generated 

20 by a cable aggregator and received by a user's set-top box as one of a number of channels. 

2. CCMPOs 2504 corresponding to each channel are sent along the cable in Header 
2505 at regular intervals (e.g., once per second). When the set-top box is turned on, it polls 
each channel, and downloads all current CCMPOs. These are stored persistently, and are 
changed only if a new CCMPO is received which differs fiom prior CCMPOs. 

^- *e user selects a channel, the set-top box addresses the associated 
CCMPO. The CCMPO may specify, for example, that content in this particular channel 
may only be accessed by subscribers to the channel. A CMPS within the set-top box 
accesses a user profile persistently stored in NVRAM and determines that the user is a 
subscriber. The CMPS deems the CCMPO rule to have been satisfied. 

The CMPS obtains an identifier for the MCMPO associated with the work 
(video) currently streaming on the channel and a key for the MCMPO. If works are 
received seriaUy on the channel (e.g., a television channel in which one work is provided at 
a time), the received MCMPO identifier may include don't care bits so that it can address 
any MCMPO currently on the channel. 
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5. The CMPS begins demuxing of Aggregate Stream 2501 (this may occur in 
paraUel with the preceding step), and obtains the MCMPO, which is encoded into an ES 
multiplexed within the Aggregate Stream (e.g.. MCMPO 2506). Although each ES within 
Aggregate Stream 2501 has been encrypted, Aggregate Stream 2501 was not encrypted 
following multiplexing. This allows the CMPS to demultiplex Aggregate Stream 2501 
without decrypting the entire Aggregate Stream. 

6. The CMPS identifies the ES which constitutes the MCMPO (e.g., ES 2503). 
The CMPS downloads one complete instance of MCMPO 2506 into an internal buffer, and 
uses the key received from CCMPO 2504 to decrypt MCMPO 2506. 

7. The CMPS determines which rules are applied by MCMPO 2506. MCMPO 
2506 might, for example, include a rule stating that the user can view the associated work 
with advertisements at a low fee, but must pay a higher fee for viewing the work without 
advertisements. 

8. The CMPS generates an options menu, and displays that menu on the screen for 
the user. The menu specifies the options, including the cost for each option. Additional 
options may be specified, including payment types. 

9. The user uses a rranote control pointing device to choose to view the work at a 
lower cost but with advertisements. The user specifies that payment can be made from an 
electronic cash budget stored in the CMPS. 

10. The CMPS subtracts the specified amount from the budget persistenUy stored 
in NVRAM, and generates and encrypts a message to a server associated with the cable. 
The message transfers the required budget to the server, either by transferring electronic 
cash, or by authorizing a financial clearinghouse to transfer fibe amount from the user's 
account to the cable provider's. This message may be sent immediately, or may be 
buffered to be sent later (e.g., when the user connects the device to the Internet). This step 
may be taken in parallel with decryption of the content) 

1 1. The CMPS obtains from MCMPO 2506 a set of keys used to decrypt the 
Elementary Streams associated with the work (e.g., ES 2502). The CMPS also obtains 
identifiers for the specific ESs to be used. Since the user has indicated that advertisements 

30 are to be included, the MCMPO identifies ESs associated with the advertisements, and 

identifies a Scene Descriptor Graph which includes advertisements. A Scene Descriptor 
Graph which does not include advertisements is not identified, and is not passed through by 
the CMPS. 

12. The CMPS passes the decrypted ESs to the MPEG-4 buffers. The normal 
process of MPEG-4 decoding, compositing and rendering then takes place. The Composite 
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and Render block outputs Start and Stop events for each object released for viewing. TTie 
CMPS monitors this information and compares it to the expected events. In particular, the 
CMPS confirms that the advertisements have been released for viewing, and that each ' 
operation has occupied approximately the expected amount of time. 

In another embodiment, a set-top box containing a CMPS (e.g., CMPS 2302 from 
FIG. 23) may have a cable input (e.g., carrying M4 Bit Streams 2114 and CMPOs 2303). 
The cable may carry multiple channels, each made up of two subH:hamiels. with one sub- 
channel carrying MPEG-4 ESs (e.g.. M4 Bit Streams 23 14). and the other sub-channel 
carrying CMPOs (e.g.. CMPOs 2303). The sub-chamiel carrying CMPOs 2303 could be 
routed directly to CMPS 2302. with the ES channel being routed to a decryption block 
(operating under control of the CMPS. e.g.. CR&D 2315), and then to the MPEG-4 buffers 
(e.g., buffers associated with Scene Descriptor Graph 2306. AVO Decode 2307 and Object 
Descriptoi. 2308). In this case, if the ESs are not encrypted, they proceed unchanged 
through the decryption block and into the buffers. This may occur, for example, if the ESs 
are being broadcast for fi-ee. with no restrictions, and/or if they are public domain 
information, and/or they were created prior to inclusion of CMPOs in the MPEG-4 
standard. 

Such an embodiment might include timing synchronization information in the 
CMPO subs:hamiel, so that CMPOs can be synchronized with the associated ESs. 

The concept of incorporating two separate streams, one consisting of control 
infonnation and connected directly to the CMPS. and the other consisting of ESs. may 
support a high degree of modularization, such that the formats of CMPOs. and particular 
types of CMPS's. may be changed without alteration to the underlying ES format. For 
example, it may be possible to change the CMPO format without the necessity for 
reformatting content ESs. To take anodier example, it may be possible to upgrade a 
Commerce Appliance by including a new or different CMPS. without the necessity for any 
changes to any of the circuitry designed to demultiplex, composite and render the content 
ESs. A user might obtain a CMPS on a smart card or other removable device, and plug that 
device into a Commerce AppUance. This could be done to customize a Commerce 
Appliance for a particular appUcation or for particular content. 

CMPS Interface to a CE Device 

A CMPS may be designed to present a standardized interfece between the general- 
purpose fonctionaHty of a consumer electronics device and any relevant CMPOs and/or 
other a and protected content For example, a CMPS could be designed to accept CI and 
encrypted ESs, and output decrypted ESs into the device's buffers. In such a case, the 
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manufacturer of the device would be able to design the device in comphance with the 
specification (e.g.. MPEG-4). without concern about comme„:e-related extensions to the 
standard, which extensions might differ fiom provider to provider. All such extensions 
would be handled by the CMPS. 
Initialization 

1- Initialization of the TMP*; 

A CMPS may be used to identify the capabilities of the Commerce 
Appliance in which a CMPS is installed. A CMPS permanentfy associated with a 
particular Commerce Appliance may have such information designed-in when the CMPS 
initially mstalled (e.g., stored in ROM 2406 shown in FIG24). A CMPS which is 
removable may be used to run an initialization opendon in order to obtain information 
about the device's capabilities. Such infomiation may be stored in a data structure stored . 
NVRAM 2425. Alternatively, some or all of such information may be gathered each time 
the device is turned on, and stored in RAM 2414. 

For example, a DVD player may or may not contain a comiection to an external 
server and/or process. A CMPO and/or other CI stored on a DVD (and/or any other format 
optical disk) inserted into a DVD (or any other format optical disk) player may include 
rules predicated on the possibiUty of outputting infomiation to a server (e.g.. content is free 
If user identification information is output), or may require a direct comiection in order for 
example, to download keys used to decrypt content In such a case, the CMPS arrangement 
may determine die hardware fimctionality which is expected by or required by the CMPO 
and compare that to the hardware actually present. If the CMPS determines that the CMPO 
and/or other CI requires a network comiection . and that the DVD player does not include 
such a comiection, the CMPS may take a variety of steps, including: (1) if the network 
comiection is required for some options but not others, causing only those options which 
are possible to be displayed to the user; (2) informing the user that necessary hardware is 
missmg; or (3) causing a gracefiil rejection of the disk, mcluding infomiing the user of the 
reason for the rejection. 

To take another example, a CMPO and/or other CI may include a business model 
which allows the user to choose among quaUty levels (or other forms of vadations of a 
given work, for example, longer length and/or greater options), with a higher price being 
charged if the user selects a higher level of quality (e.g.. music may be played at low 
resolution for free, but requires a payment in order to be played at a higher resolution). In 
such a case, the Commerce Appliance may not include loudspeakers which ar« capable of 
outputting sound at the higher resolution. The CMPS arrangement preferably identifies 
this situation, and either eliminates the higher resolution output as an option for the user or 
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informs the user that this option costs more but provides no additional benefit given the 
Conunerce Appliance's current functionality or given the Commerce Appliance not being 
docked in a user arrangement that provides higher quality loudspeakers. 

If the Commerce Appliance may be hooked up to external devices (e.g., 
loudspeakers, display, etc.), the CMPS will require some mechanism for identifying and 
registering such devices. Each device may be used to make standard ID and capability 
information available at all times, thereby allowing the CMPS to poU all connected devices 
at regular intervals, including, for example, authenticating CMPS arrangements within one 
or more of each such connected devices. Using another approach, aU devices could be used 
to output CMPS identification information upon power-on, with later connected devices 
being used to output such information upon establishment of the connection. Such 
identification information may take the form, for example, of authentication information 
provided under the "five company arrangement", such authentication methods are herein 
incorporated by reference. 

As discussed earlier, a Commerce Appliance may be connected to multiple devices 
each containing its own CMPS arrangement (e.g., a DVD player may be connected to a 
digital TV) In such cases, the CMPSs must be able to initiate secure communication (e. g.. 
using a scheme, for example, like the "five company proposal" for IEEE 1394 serial bus) 
and determine how the CMPSs will interact vnth respect to content communication 
between CMPSs and, in certain embodiments, regarding cooperative governance of such 
content such as describing in the incorporated Shear patent application. In one 
embodiment, tiie first CMPS arrangement to receive content might govern the control 
process by downloading an mitial CMPO and/or other CI, and display one or more of the 
rules to the user, etc. The second CMPS arrangement might recognize that it has no further 
role to play, either as a result of a communication between the two CMPS arrangements, or 
as a result of changes to the content sti^ created by the first CMPS arrangement (which 
decrypted the content, and may have allowed demuxing, composition and rendering, etc.) 

The relationship between upstream and downstream CMPSs arrangements may be 
complicated if one device handles certain aspects of MPEG-4 rendering, and tiie otiier 
handles other aspects. For example, a DVD player might handle demuxing and buffering, 
transferring raw ESs to a digital TV, which then handles composition and rendering, as 
well as display. In such a case, there might be no back-chamiel from the composition and 
rendering block to the upstream CMPS arrangement. CMPS arrangements are preferably 
designed to handle stand-alone cases (a DVD (or any oUier optical disk) player with a 
CMPS arrangement attached to a dumb TV with no CMPS). multiple CMPS arrangement 
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cases in which one CMPS arrangement handles all of the processing (a DVD (or other 
opUcal disk) player which handles everything through composition and rendering, with a 
video stream output to the digital TV (in one non-limiting example, via an IEEE 1349 
serial bus) (that output stream would be encrypted as per the "five company proposal" for 
copy protection using IEEE 1394 serial bus transmission)) and/or shared processing 
between two or more CMPSs arrangements regarding some, or in certain cases, all, of such 
processing. 

2. Initialization of a particular content stream . 

The CMPS may be designed so that it can accept initialization information 
which initializes the CMPS for a particular content stream or channel. This header, which 
may be a CMPO and/or other CI, may contain information used by the CMPS to locate 
and/or interpret a particular content stream as well as CI associated with that stream. This 
initial header may be received through a sideband channel, or may be received as a CI ES 
such as a CMPO ES. 

In one example, shown in FIG. 26, Header CMPO 2601 may include the following 
information: 

(a) Stream/Object/CMPO ID 2602, which identifies the content 
streams/objects governed by Header CMPO 2601 and/or identification of CMPOs 
associated with each such content stream or object. 

In one embodiment. Header CMPO 2601 identifies other CMPOs which contam 
rules and keys associated with particular content streams. In another embodiment. Header 
CMPO 2601 directly controls all content streams, by incorporating the keys and rules 
associated with such streams. In the latter case, no other CMPOs may be used. 

In one embodiment. Header CMPO 2601 may be one or more CMPOs, CCMPOs, 
MCMPOs, and/or other CI. 

(b) One or CMPO Keys 2603 for decrypting each identified CMPO. 

(c) Work-Level Control 2604, consisting of basic control information 
associated with the work as a whole, and therefore potentially applicable to all of the 
content streams which make up the work. TTiis basic control information may include rules 
governing the work as a whole, including options to be presented to the user. 

(d) In one embodiment of this embodiment, a header CMPO may be 
updatable to contain User/Site Information 2605 regarding a particular user or site currently 
authorized to use certain content, as well as one or more rule sets under which the user has 
gained such authorization. A header CMPO associated with a work currently being viewed 
may be stored in RAM or NVRAM. This may include updated information. In one 
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embodiraent, the CMPO may also store header CMPOs for certain works viewed in the 
past. In one embodiment, header CMPOs may be stored in non-secure memory, with 
information sufficient to identify and authenticate that each header CMPO had not been 
changed. 

In one such header CMPO embodiment of this embodiment, the header CMPO 
operates as follows: 

(a) The header CMPO is received by a CMPS arrangement. In the case of 
previously unreceived content which has now become available, the header CMPO may be 
received at an input port. In the case of content which is already available, but is not 
currently being used (e.g., a set-top box with 500 channels, of which either 0 or 1 are being 
displayed at any given time), CCMPOs for each channel may be buffered by the CMPS 
arrangement for possible use if the user invokes particular content (e.g., switches to a 
particular channel). 

In either case, the header CMPO must include information which allows a CMPS 
1 5 arrangement to identify it as a header CMPO. 

(b) The CMPS arrangement obtains business-model information held in the 
clear in the header CMPO. Business-model information may include, for example, a 
statement that content can be viewed for free if advertisements are included, or if the user 
authorizes Nielson-type information, user and/or audience measurement information, for 

20 example, content may be output to a server or otherwise copied once, but only at a price. 

(c) The CMPS arrangement either accepts the business model, if the user 
has authorized it to accept certain types of models (e.g., the user has programmed the 
CMPS arrangement to always accept play v«th advertisements for free), rejects the 
business model, if the user has instructed that the particular model always be rejected, or 

25 displays the business model to the user (e.g., by presenting options on the screen). 

(d) If a business model has been accepted, the CMPS arrangement then 
decrypts the remainder of the header CMPO. If the Commerce Appliance contains a live 
output connection to an external server (e.g., Internet connection, back-channel on a set-top 
box, etc.), and if latency problems are handled, decryption of these keys can be handled by 

30 communicating with the external server, each side authenticating the other, establishment 

of a secure channel, and receipt of a key from the server. If the Commerce Appliance is not 
at least occasionally connected to an external server, decryption may have to be based on 
one or more keys securely stored in the Commerce Appliance. 

(e) Once a header CMPO has been decrypted, the CMPS arrangement 
35 acquires information used to identify and locate the streams containing the content, and 
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keys which are used to decrypt either the CMPOs associated with the content, or to directly 
decrypt the content itself. 

(f) In one embodiment of this header embodiment, the header CMPO may 
contain a data structure for the storage of information added by the CMPS arrangement. 
5 Such information may include the following: 

(1) Identification of user and/or Commerce Appliance and/or CMPS 
arrangement. In this embodiment, such information may be stored in a header CMPO in 
order to provide an audit trail in the event the work (including the header CMPO) is 
transferred ( this only works if the header CMPO is transferred in a writable form). Such 

10 • information may be used to allow a user to transfer the work to other Commerce 

Appliances owned by the user without the payment of additional cost, if such transfers are 
allowed by rule information associated with the header CMPO. For example, a user may 
have a subscription to a particular cable service, paid for in advance by the user. When a 
CMPS arrangement downloads a header CMPO fix)m that cable service, the CMPS 
1 5 arrangement may store the user's identification in the header CMPO. The CMPS 

arrangement may then require that the updated header CMPO be included if the content is 
copied or transferred. The header CMPO could include a rule stating that, once the user 
information has been filled in, the associated content can only be viewed by that user, 
and/or by Commerce Appliances associated with that user. This would allow the user to 
make multiple copies of the work, and to display the work on multiple Commerce 
Appliances, but those copies could not be displayed or used by non-authorized users and/or 
on non-authorized Commerce Appliances. The header CMPO might also include a rule 
stating that the user information can only be changed by an authorized user (e.g., if user 1 
transfers the work to user 2, user 2's CMPS arrangement can update the user information in 
the header CMPO, thereby allowing user 2 to view the work, but only if user 2 is also a 
subscriber to the cable channel). 

(2) Identification ofparticular rules options governing use. Rule 
sets included in header CMPOs may include options. In certam cases, exercise of a 
particular option might preclude later exercise of a different option. For example, a user 
might be given the choice to view an unchanged work for one price, or to change a work 
and view the changed work for a higher price. Once the user decides to change the work 
and view the changed work, this choice is preferably stored in the header CMPO, since the 
option of viewing the original unchanged work at the lower price is no longer available. 
The user might have fiirther acquired the right, or may now be presented with the option for 
the right, to further distribute the changed work at a mark-up in cost resulting in third party 
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derived revenue and usage infomation flowing to both the user and the original work 
stakeholder(s). 

(3) Historical usage information. The header CMPO may include 
information relating to the number and types of usages. For example, if the underlying 
work is copied, the header CMPO may be updated to reflect the fact that a copy has been 
made, since a rule associated with the woric might allow only a single copy (e.g., for 
backup and/or timeshifting purposes). To take another example, a user might obtain the 
right to view a work one time, or for a certain number of times. The header CMPO would 
then be updated to reflect each such use. 

Usage information may be used to determine if additional uses are authorized by 
rules associated with the header CMPO. Such information may also be used for audit 
purposes. Such information may also be provided as usage information exhaust, reported 
to an external server. For example, a rule may specify that a work may be viewed for free, 
but only if historical usage information is downloaded to a server. 
^ ^ Content Management Protection Objects (CMPO) 

The Content Management and Protection Object ("CMPO") is a data structure 
which includes information used by the CMPS to govern use of certain content. A CMPO 
may be formatted as a data structure specified by a particular standard (e.g., an MPEG-4 
ES), or may be formatted as a data structure not defined by the standard. If the CMPO is 
20 formatted as a data structure specified by the standard, it may be received in the channel 

utilized by the standard (e.g., as part of a composite MPEG-4 stream) or may be received 
through some other, side-band method. If the CMPO is formatted as a data structure not 
specified by the relevant standard, it is provided and decoded using some side-band 
method, which may include receipt through the same port as formatted content and/or may 
25 include receipt through a separate port 

Content may be controUed at virttially any level of granularity. Three exemplary 
levels will be discussed herein: "channel," "work," and "object" 

A "channel" represents an aggregation of works. The works may be available for 
selection by the user (e.g., a web site, or a video library) or may be received seriaUy (e.g., a 
30 cable television channel). 

A "work" represents a single audio-visual, textual or other work, intended to be 
consumed (viewed, read, etc.) by a user as an integrated whole. A work may, for example, 
be a movie, a song, a magazine article, a multimedia product such, for example, as 
sophisticated videogame. A woric may incorporate other works, as, for example, in a 
multimedia work which incorporates songs, video, text, etc. In such a case, rights may be 
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associated 

An "object" represents a separately addressable portion of a work. An object may 
be. for example, an individual MPEG-4 AVO. a scene descriptor graph, an object 
descriptor, the soundtrack for a movie, a weapon in a videogame, or any other logically 
definable portion. 

Content may be controlled at any of these levels (as well as intermediate levels not 
discussed herein). TTie prefened embodiment mechanism for such control is a CMPO or 
CMPO anangement (which comprises one or more CMPOs. and if plural, then plural, 
cooperating CMPOs). CMPOs and CMPO arrangements may be organized hierarchically, 
with a Channel CMPO anangement imposing rules applicable to all contained works, a 
MCMPO or an SGCMPO imposing rules applicable to all objects within a work, and a 
CMPO arrangement imposing rules appUcable to a particular object. 

In one embodiment, illustrated in FIG. 27, a CMPS may download CCMPO 2701. 
CCMPO 2701 may include one or more Rules 2702 applicable to all content in the 
chamiel, as well as one or more Keys 2703 used for decryption of one or more MCMPOs 
and/or SGCMPOs. MCMPO 2704 may include Rules 2705 applicable to a single work 
and/or works, one or more classes and/or more users and/or user classes, and may also 
include Keys 2706 used to decrypt CMPOs. CMPO 2707 may include Rules 2708 
applicable to an individual object, as weU as Key 2709 used to decrypt the object. 

As long as all objects are subject to control at some level, there is no requirement 
that each object be individually controUed. For example. CCMPO 2701 could specify a 
single rule for viewing content contained in its channel (e.g., content can only be viewed by 
a subscriber, who is then might be fiee to redistribute the content with no further obligation 
to the content provider). In such a case, rules would not necessarily be used for MCMPOs 
(e.g. Rules 2705). SGCMPOs, or CMPOs (e.g.. Rules 2708). In one embodiment, 
MCMPOs, SGCMPOs. and CMPOs could be dispensed widi, and CCMPO 2701 could 
include all keys used to decrypt aU content, or could specify a location where such keys 
could be located. In another embodiment, CCMPO 2701 would supply Key 2703 used to 
decrypt MCMPO 2704. MCMPO 2704 might include keys used to decrypt CMPOs (e.g.. 
Keys 2706), but might include no additional Rules 2705. CMPO 2707 might include Key 
2709 used to decrypt an object, but might include no additional Rules 2708. In certain 
embodiments, there may be no SGCMPOs. 

A CMPO may be contained within a content data structure specified by a relevant 
standard (e.g., the CMPO may be part of a header in an MPEG-4 ES.) A CMPO may be 
contained within its own, dedicated data structure specified by a relevant standard (e.g.. a 
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CMPO ES). A CMPO may be contained within a data structure not specified by any 
content standard (e.g., a CMPO contained within a DigiBox). 
A CCMPO may include the following elements: 

(a) ID 2710. This may take the following form: <channel IDx CMPO 
typeXCMPO IDxversion number>. In the case of hierarchical CMPO organization (e.g., 
CCMPOs controUing MCMPOs controlling CMPOs), CMPO ID 2711 can include one 
field for each level of the hierarchy, thereby allowing CMPO ID 271 1 to specify the 
location of any particular CMPO in the organization. ID 2710 for a CCMPO may, for 
example, be 123-000-000. ID 2712 for a MCMPO of a work within that chamiel may, for 
example, be 123-456-000, thereby allowing the specification of 1,000 MCMPOs as 
controlled by the CCMPO identified as "123." CMPO ID 271 1 for a CMPO associated 
with an object within the particular work may, for example, be 123-456-789, thereby 
allowing the specification of 1 ,000 CMPOs as associated with each MCMPO. 

This method of specifying CMPO IDs thereby conveys the exact location of any 
CMPO within a hierarchy of CMPOs. For cases in which higher levels of the hierarchy do 
not exist (e.g., a MCMPO with no associated CCMPO), the digits associated with that level 
of the hierarchy may be specified as zeroes. 

(b) Rules 2702 applicable to all content in the channel. These may be self- 
contained rules, or may be pointers to rules obtainable elsewhere. Rules are optional at this 

20 level. 

(c) Infonnation 2713 designed for display in the event the user is unable to 
comply with the rules (e.g., an advertisement screen informing the user that a subscription 
is available at a certain cost, and including a list of content available on the channel). 

(d) Keys 2703 for the decryption of each MCMPO controlled by this 
CCMPO. In one embodiment, the CCMPO mcludes one or more keys wWch decrypt all 
MCMPOs. In an alternate embodiment, the CCMPO includes one or more specific keys 
for each MCMPO. 

(e) A specification of a CMPS Type (2714), or of hardware/software 
necessary or desirable to use the content associated with this channel. 

The contents of a MCMPO may be similar to those of a CCMPO, except that the 
MCMPO may include rules applicable to a single work, and may identify CMPOs 
associated with each object 

The contents of each CMPO may be similar to those of the MCMPO, except that 
the CMPO may mclude rules and keys appUcable to a single object. 
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The contents of an SGCMPO may be similar to those of the CCMPO, except that 
the MCMPO may mclude rules applicable to only certain one or more classes of rights, 
certain one or more classes of works, and/or to one or more certain classes of users and/or 
user arrangements (e.g. CMPO arrangements and/or their devices). 

In another embodiment, shown in FIG. 28, CMPO Data Structure 2801 may be 
defined as follows: 

CMPO Data Structure 2801 is made up of elements. Each element includes a self- 
contained item of information. The CMPS parses CMPO Data Structure, one element at a 
time. 

Type Element 2802 identifies the data structure as a CMPO, thereby allowing the 
CMPS to distinguish it from a content ES. In an exemplary embodiment, this element may 
include 4 bits, each of which may be set to " 1" to indicate that the data structure is a 
CMPO. 

The second element is CMPO Identifier 2803, which is used to identify this 
particular CMPO and to convey whether the CMPO is part of a hierarchical organization of 
CMPOs and, if so, where this CMPO fits into that organization. 

CMPO Identifier 2803 is divided into four sub-elements, each of three bits. These 
are shown as sub-elements A, B, C and D. The first sub-element (2803 A) identifies the 
CMPO type, and indicates whether the CMPO is governed or controUed by any other 
CMPO: 

1 00: this is a top-level CMPO (associated vnth a channel or an aggregation of 
works) and is not controlled by any other CMPO. 

010: this is a mid-level CMPO (associated with a particular work) and is not 
controlled by any other CMPO. 

110: this is a mid-level CMPO, and is controlled by a top-level CMPO. 

001: this is a low-level CMPO (associated with an object within a work) and is not 
controlled by any other CMPO. This case wiU be rare, since a low-level CMPO will 
ordinarily be controlled by at least one higher-level CMPO. 

01 1: this is a low-level CMPO, and is controlled by a raid-level CMPO, but not by 
a top-level CMPO. 

1 11 : this is a low-level CMPO, and is controlled by a top-level CMPO and by a 
mid-level CMPO. 

The second sub-element of CMPO ID 2803 (sub-element B) identifies a top-level 
CMPO. In the case of a top-level CMPO, this identifier is assigned by the creator of the 
CMPO. In the case of a mid-level or low-level CMPO which is controlled by a top-level 
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CMPO. this sub-element contains the identification of the top-level CMPO which performs 
such control. In the case of a mid-level or low-level CMPO which is not controlled by a 
top-level CMPO, this sub-element contains zeroes. 

The third sub-element of CMPO ID 2803 (sub-element C) identifies a mid-level 
CMPO. In the case of a top-level CMPO. this sub-element contains zeroes. In the case of 
a mid-level CMPO, this sub-element contains the identification of the particular CMPO. In 
the case of a low-level CMPO which is controlled by a mid-level CMPO, this sub-element 
contains the identification of the mid-level CMPO which performs such control. In the 
case of a low-level CMPO which is not controlled by a mid-level CMPO. this sub-element 
contains zeroes. 

The fourth sub-element of CMPO ID 2803 (sub-element D) idenUfies a low-level 
CMPO. In the case of a top-level or mid-level CMPO, this sub-element contains zeroes. 
In the case of a low-level CMPO, this sub-element contains the identification of the 
particular CMPO. 

Following the identifier element is Size Element 2804 indicating the size of the 
CMPO data structure. This element contains the number of elements (or bytes) to the final 
element in the data structure. This element may be rewritten if alterations are made to the 
CMPO. The CMPS may use this size information to determine whether the element has 
been altered without permission, since such an alteration might result in a different size. 
For such purposes, the CMPS may store the information contained in this element in a 
protected database. This information can also be used to establish that the entire CMPO 
has been received and is available, prior to any attempt to proceed witii processing. 

Following Size Element 2804 are one or more Ownership/Control Elements 
containing ownership and chain of control information (e.g., Ownership/Control Elements 
2805, 2806 and 2807). In the first such element (2805), die creator of the CMPO may 
include a specific identifier associated with that creator. Additional participants may also 
be identified in following elements (e.g., 2806, 2807). For example. Element 2805 could 
identify the creator of the CMPO. Element 2806 could identify the publisher of the 
associated work and Element 2807 could identify the author of the work. 

A specific End Element 2808 sequence (e.g., 0000) indicates the end of the chain of 
ownership elements. If this sequence is encountered in tiie first element, this indicates tiiat 
no chain of ownership information is present 

Chain of ownership information can be added, if rules associated witii CMPO 2801 
permit such additions. If, for example, a user purchases tiie work associated with CMPO 
2801, tiie user's identification may be added as a new element in the chain of ownership 
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elements (e.g., a new element following 2807. but before 2808). This may be done at the 
point of purchase, or may be accomplished by the CMPS once CMPO 2801 is encountered 
and the CMPS deteimines that the user has purchased the associated work. In such a case, 
the CMPS may obtain the user identifier from a data structure stored by the CMPS in 
5 NVRAM. 

Following the ownership element chain are one or more Handling Elements (e.g^ 
2809, 2810) indicating chain of handling. These elements may contain the identificaUon of 
any CMPS which has downloaded and decoded CMPO 2801, and/or may contain the 
identification of any user associated with any such CMPS. Such information may be used 

10 for audit purposes, to allow a trail of handling in the event a work is detemiined to have 

been circulated improperly. Such information may also be reported as exhaust to a 
clearinghouse or central server. Chain of handling information preferably remams 
persistent untU reported. If the number of elements required for such infomiation exceeds a 
specified amount (e.g., twenty separate user identifiers), a CMPS may refuse to allow any 

1 5 further processing of CMPO 280 1 or the associated work until the CMPS has been 

connected to an external server and has reported the chain of handling information. 

The last element in the chain of handling elements (e.g., 28 1 1) indicates the end of 
this group of elements. The contents of this element may, for example, be all zeroes. 

Following the chain of handling elements may be one or more Certificate Elements 

20 (e.g., 2812, 28 13) containing or pointing to a digital certificate associated with this CMPO. 

Such a digital certificate may be used by the CMPS to authenticate the CMPO. The final 
element in the digital certificate chain is all zeroes (2814). If no digital certificate is 
present, a single element of all zeroes exists in this location. 

Following the Certificate Elements may be a set of Governed Object Elements (e.g., 

25 2815, 2816, 2817, 2818) specifying one or more content objects and/or CMPOs which may 

be governed by or associated with CMPO 2801. Each such governed object or CMPO is 
identified by a specific identifier and/or by a location wh«e such object or CMPO may be 
found (e.g., these may be stored in locations 2815 and 2817). Following each such 
identifier may be one or more keys used to decrypt such CMPO or object (e.g., stored in 

30 locations 2816 and 2818). The set of identifiers/keys ends with a termination element 

made up of all zeroes (2819). 

Following the set of elements specifying identifiers and/or keys may be a set of 
Rules Elements (e.g., 2820, 2821, 2822) specifying rules/controls and conditions associated 
with use of the content objects and/or CMPOs identified in the Governed Objects chain 

35 (e.g., locations 2815 and 2817). Exemplary rules are described below. Elements may 
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contain explicit rules or may contain pointers to rules stored elsewhere. Conditions may 
mclude particular hardware resourees necessary to use associated content objects or to 
satisfy certain rules, or particular types of CMPS's which are necessary or preferred for use 
of the associated content objects. 

Following the rules/controls and conditions elements may be a set of Infomiation 
Elements 2823 containing information specified by the creator of the CMPO. Among other- 
contents, such infomiation may include content, or pointers to content, programming, or 
pointers to programming. 

The CMPO ends with Final Termination Element 2824. 

In one embodiment, the rules contained in Rules Elements 2820-2822 of CMPO 
2801 may include, for example, the following operations: 

(1) Play. This operation allows the user to play the content (though not to 
copy it) without restriction. 

(2) Navigate. This allows the user to perform certain types of navigation 
functions, including fast forward/rewind, stop and search. Search may be indexed or 
unindexed. 

(3) Copy. Copy may be allowed once (e.g., time-shifting, archiving), may - 
be allowed for a specified number of times and/or may be allowed for limited period of 
time, or may be allowed for an unlimited period of time, so long as other rules, including 
relevant budgets, are not violated or exceeded. A CMPS arrangement may be designed so 
that a Copy operation may cause an update to an associated CMPO (e.g., including an 
indication that the associated content has been copied, identifying the date of copying and 
the site responsible for making the copy), without causing any change to any applicable 
content object, and m particular without requiring that associated content objects be 
demuxed, decrypted or decompressed. In the case of MPEG-4. for example, this may 
require the following multi-stage demux process: 

(i) the CMPS arrangement receives a Copy instruction from the 
user, or from a header CMPO. 

(ii) CMPO ESs associated with the MPEG-4 stream which is to be 
copied are separated from the content stream in a fu^t demux stage. 

(iii) CMPOs are decrypted and updated by the CMPS arrangement. 
The CMPOs are then remuxed with the content ESs (which have never been demuxed from 
each other), and the entire stream is routed to the output port without further alteration. 

This process allows a copy operation to take place without requiring that the 
content streams be demuxed and decrypted. It requires that the CMPS arrangement include 
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two outputs: one output connected to the digital output port (e.g., FIG. 23 line 23 1 6, 
connecting to Digital Output Port 2317). and one output connected to the MPEG-4 buffers 
(e.g., FIG. 23. lines 23 10. 23 1 1. 2312), with a switch designed to send content to one 
output or the other (or to both, if content is to be viewed and copied simultaneously) (e.g.. 
Switch 2319). Switch 2319 can be the only path to Digital Output Port 2317 thereby 
allowing CMPS 2302 to exercise direct control over that port, and to ensure ^t content is 
never sent to that port unless authorized by a control. If Digital Output Port 23 1 7 is also 
the connector to a digital display device. CMPS 2302 will also have to authorize content to 
be sent to that port even if no copy operation has been authorized. 

In one example embodiment, the receiving device receiving the information 
through Digital Output Port 2317 may have to authenticate with the sending device (e.g., 
CMPS 2302). Authentication may be for any characteristic of the device and/or one or ' 
more CMPSs used in conjunction with that device. Thus, for example, a sending appliance 
may not transmit content to a storage device lacking a compatible CMPS. 

In another non-limiting example, CMPS 2302 can incorporate session encryption 
functionality (e.g., the "five company arrangement" ) which establishes a secure channel 
from a sending interface to one or more external device interfaces (e.g., a digital monitor), 
and provided that tiie receivmg interface has autiienticated with tiie sending interface, 
encrypts the content so that it can only be decrypted by one or more authenticated 1 394 
device interfaces. In that case. CMPS 2302 would check for a suitable IEEE 1394 serial 
bus interface . and would allow content to flow to Digital Output Port 23 1 7 only if (a) an 
authorized Play operation has been invoked, a secure channel has been established witii the 
device and the content has been session-encrypted, or (b) an authorized Copy or Retransmit 
operation has been invoked, and the content has been treated as per the above description 
(i.e., the CMPO has been demuxed, changed and remuxed, the content has never been 
decrypted or demuxed). 

This is only possible if CMPOs are separately identifiable at an early demux stage, 
which most likely requires that they be stored in separate CMPO ESs. If the CMPOs are ' 
stored as headers in content ESs, it may be impossible to idemily tiie CMPOs prior to a full 
demux and decrypt operation on the entirety of tiie stream. 

(4) Change. The user may be autiiorized to change tiie content. 

(5) Delete. This command allows tiie user to delete content which is stored 
in tiie memory oftiie Consumer Appliance. This operation operates on the entire work. If 
tiie user wishes to delete a portion of a work, tiie Change operation must be used. 

(6) Transfer. A user may be autiiorized to transfer a work to a tiiird party. 
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This differs from the Copy operation in that the user does not retain the content or any 
rights to the content. The Transfer operation may be canied out by combining a Copy 
operation and a Delete operation. Transfer may require alteration of the header CMPO 
associated with the work (e.g.. adding or altering an Ownership/Control Element, such as 
Elements 2805-2807 of FIG. 28), so as to associate rights to the work with the third party. 
These basic operaHons may be subject to modifications, which may include: 

I. Payment. Operations may be conditioned on some type of user 
payment. Payment can take the form of cash payment to a provider (e.g., credit card, 
subtraction from a budget), or sending specified information to an external site (e.g.,' 
Nielson-type information). 

h. Quality of Service. Operations may specify particular quality of 
service parameters (e.g., by specifying a requested QoS in MPEG-4), including: requested 
level of decompression, requested/required types of display, rendering devices (e.g., higher 
quality loudspeakers, a particular type of game controller). 

iii. Time. Operations may be conditioned such that the operation is 
only allowed after a particular time, or such that the price for the operation is tied to the 
time (e.g., real-time information at a price, delayed information at a lower price or free, 
e.g., allowing controlled copies but only after a particular date). 

iv. Display of particular types of content. Operations may be 
conditioned on the user authorizing display of certain content (e.g., the play operation may 
be free if the user agrees to allow advertisements to be displayed). 

In all of tiiese cases, a rule may be modified by one or more other rules. A rule may 
specify that it can be modified by other rules or may specify tiiat it is umnodifiable. If a 
rule is modifiable, it may be modified by rules sent from other sources. Those rules may 
be received separately by the user or may be aggregated and received together by the user. 

Data types which may be used in an exemplary MPEG-4 embodiment may include 
the following: 

a. CMP Data Stream. 

The CMP-ds is a new elementary stream type tiiat has all of the properties of an 
elementary stream including its own CMPO and a reference in the object descriptors. Each 
CMP-ds stream has a series of one or more CMP Messages. A CMP_Message has four 
parts: 



1. Count: (!...«] CMPS types supported by this IP ES. Multiple CMPS 
systems may be supported, each identified by a unique type. (There may have 
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to be a central registry of types.) 

2. CMPS.typeJdentifiers: [!...«] identifiers, each with an offset in the 
stream and a length. The offset points to the byte in the CMPO where the data 
for that CMPS type is found. The length is the length in bytes of this data. 

3. Data segments: One segment for each of the n CMPS types encoded in a 
format that is proprietary to the CMPS supplier. 

4. CMP_Message_URL: That references another CMP_Message. (TOsisin 
keeping with the standard of using URLs to point to streams.) 

b. CMPO. 

The CMPO is a data structure used to attach detailed CMP control to individual 
elementary streams. Each CMPO contains: 

1. CMPO_n): An identifier for the content under control. This identifier must 
uniquely identify an elementary stream. 

2. CMPO_count: [I.../1] CMPS types supported by this CMPO. 

3. CMPS_type_identifiers: [!...«] identifiers, each with an offset in the 
stream and a length. TTie offset points to the byte m the CMPO where the data 
for that CMPS type is found. The length is the length in bytes of this data. 

4. Data segments: « data segments. Each data segment is in a format that is 
proprietary to the CMPS supplier. 

5. CMPO_URL: An optional URL that references an additional CMPO that 
adds information to the information in this CMPO. (This is a way of 
dynamically adding support for new CMPSs.) 

c. Feedback Event 

The feedback events come in two forms: start and end. Each feedback event 
contains three pieces of information: 

1. £Iementary_stream_ID 

2. Time: in presentation time 

3. Object_instance_number 
User Interface. 

Commerce Appliance 2301 may include User Interface 2304 designed to convey 
control-related information to the user and to receive commands and information from the 
user. This interface may include special purpose displays (e.g., a light which comes on if a 
current action requires payment), special purpose buttons (e.g.. a button which accepts the 
payment or other tenns required for display of content), and/or visual information presented 



on screen. 
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Example of Operation in an MPEG-4 Context 

1. Userselectsaparticularworkorchannel. The user may, for example, use a 
remote control device to tune a digital TV to a particular chamiel. 

2. Selection ofthe channel is communicated to a CMPS arrangement, which uses 
the mfonnation to either download a CCMPO or to identify a previously downloaded 
CCMPO (e.g.. if the CMPS anangement is contained in a set-top box. the set-top box may 
automatically download CCMPOs for eveo^ channel potentially reachable by the box) 

3. The CMPS arrangement uses the CCMPO to identify rules associated with all 
content found on the chamxel. For example, the CCMPO may specify that content may 
only be viewed by subscribers, and may specify that, if the user is not a subscriber an 
advertisement screen should be put up inviting the user to subscribe. 

4. Once mles specified by the CCMPO have been satisfied, Ae CCMPO specifies 
the location of a MCMPO associated with a particular work which is available on the 
ch„e channel CMPO may also supply oneor more keys used for deci^^^^^ 

5. The CMPS arrangement downloads the MCMPO. In the case of an MPEG-4 
embodiment, the MCMPO may be an Elementaty Stream. Tins Elementary Stream must be 
Identifiable at a relatively early stage in the MPEG-4 decoding process. 

6. The CMPS arrangement dec^pts the MCMPO, and determines the rules used to 
accassandusethecontent. The CMPS arrangement presents the user with a set of options 
mcluding the ability to view for fiee .^dth advertisements, or to view for a price without ' 
advertisements. 

7. TheuserseIectsviewforfreewithadvertisements,e.g.,byhighlightingand 
selecting an option on the screen using a remote control device. 

8. The CMPS arrangement acquires one or more keys from the MCMPO and uses 
those keys to dec^^pt the ESs associate with the video. TTie CMPS arrangement identifies 
two possible scene descriptor graphs, one with and one without advertisements. The CMPS 
anangement passes the scene descriptor graph with advertisements through, and blocks the 

Other scene descriptor graph. 

9. The CMPS arrangement monitors the composite and render block, and checks to 
determine that the advertisement AVOs have actually been released for viewing If the 
CMPS anangement detennines that those AVOs have not been .^leased for viewing it puts 
up an en-or or warning message, and tenninates further deception. 

CMPS Rights Management In Provider And Distribution Chains 

In addition to consumer arrangements, in other embodiments one or more CMPSs 
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may be used m creating, capturing, modifying, augmenting, animating, editing, excerpting 
extracting, embedding, enhancing, conecting, fingerprinting, watermarking, and/or ' 
rendermg digital information to associate rules with digital irrformation and to enforce those 
rules throughout creation, production, distribution, display and/or performance processes 

In one non-hmiting example, a CMPS, a non-exhaustive example of which may 
include a least a secure portion of a VDE node as described in the aforementioned Ginter et 
al., patent specification, is incorporated in video and digital cameras, audio microphones 
recording, playback, editing, and/or noise reduction devices and/or any other digital device 
Images, video, and/or audio, or any other relevant digital infomiation may be captured ■ 
recorded, and persistenUy protected using at least one CMPS and/or at least one CMPO 
CMPSs may interact with compression/decompression, encryption/decryption. DSP. digital 
to analog, analog to digital, and communications hardware and/or software components of 
these devices as well. 

In another non-exhaustive example, computer animation, special effects, digital 
editing, color correcting, noise reduction, and any other applications tiiat create and/or use 
digital infomiation may protect and/or manage rights associated with digital information 
using at least one CMPS and/or at least one CMPO. 

Another example includes the use of CMPSs and/or CMPOs to manage digital 
assets m at least one digital library, asset store, film and/or audio libraries, digital vaults 
and/or any other digital content storage and management means. 

In accordance with tiie present applications, CMPSs and/or CMPOs may be used to 
manage nghts in conjunction with the public display and/or performance of digital works 
In one non-exhaustive example, flat panel screens, displays, monitors, TV projectors LCD 
projectors, and/or any other means of displaying digital information, may incorporate at 
least one hardware and/or software CMPS instance that controls the use of digital works A 
CMPS may allow use only in conjunction with one or more digital credentials, one example 
of which is a digital certificate, that warrant that use of the digital infomiation will occur in 
a settmg, location, and/or other context for public display and/or perfomiance. Non-limiting 
examples of said contexts include theaters, bars, clubs, electronic biUboards. electronic 
displays in public areas, or TVs in airplanes, ships, trains and/or otiier public conveyances 
niese credentials may be issued by trusted tiiird parties such as certifying autiiorities non- 
exhaustive examples of which are disclosed in tiie aforementioned Ginter 71 2 patent 
application. 
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Additional MPEG-4 Embodiment Information 

This woric is based on the MPEG-4 description in the version 1 Systems Committee 
Draft (CD), currently the most complete description of the evolving MPEG-4 standard. 

This section presents the structural modificaUons to the MPEG-4 player architecture 
and discusses the data lines and the concomitant functional changes. Figure 23 shows the 
functional components ofthe original MPEG-4 player. Content arrives at Player 2301 
packaged into a serial stream (e.g.. MPEG-4 Bit Stream 23 14). It is demultiplexed via a 
sequence of three demultiplexing stages (e.g.. Demux 2305) into elementary streams. 
There are three principle types of elementary streams: AV Objects (AVO), Scene 
Descriptor Graph (SDG). and Object Descriptor (OD). These streams are fed into 
respective processing elements (e.g., AVO Decode 2307, Scene Descriptor Graph 2306 
Object Descriptors 2308). Th. AVOs are the multimedia content streams such as audio', 
video, synthetic graphics and so on. They are processed by the player's 
compression/coding subsystems. The scene descriptor graph stream is used to build the 
scene descriptor grapL This tells Composite and Render 2309 how to construct the scene 
and can be thought of as the "script." The object descriptors contain description information 
about tiie AVOs and the SD-graph updates. 

To accommodate a CMPS (e.g., CMPS 2302) and to protect content effectively, the 
player stiiicture must be modified in several ways: 

• Certain data paths must be rerouted to and from the CMPS 

• Certain buffers in the SDG, AVO decode and Object descriptor modules must 

be secured 

• Feedback paths from the user and the composite and render units to the CMPS 
must be added 

In order for CMPS 2302 to communicate with the MPEG-4 mtit, and for it to 
effectively manage contem we must specify the CMPO structiue and association protocols 
and we must define the communication protocols over the feedback systems (from the 
compositor and the user.) 

The structural modifications to the player are shown in Figure 23. The principal 
changes are: 



• All elementary streams are now routed through CMPS 2302. 

• Direct communication path between Demux 2305 and CMPS 2302. 

• A required "Content Release and Decrypt" Module 23 1 5 in CMPS 2302. 
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. The addition of a feedback loop (e.g.. Line 2313) from Composite and Render 
2309 to CMPS 2302. 

• Bi-directional user interaction directly with the CMPS 2302, through Line 2316 
Furthermore, for M4v2P, CMP-objects are preferably associated with aU elementary 
streams. Elementary streams that the author chooses not to protect are still marked by an 
"unprotected content" CMPO. The CMPOs are the primary means of attaching rules 
information to the content Content here not only refers to AVOs, but also to the scene 
descriptor graph. Scene Descriptor Graph may have great value and will thus need to be 
protected and managed by CMPS 2302. 

The direct path from Demux 2305 to CMPS 2302 is used to pass a CMPS specific 
header, that potentially contains business model information, that communicates business 
model information at the beginning of user session. This header can be used to initiate user 
identification and authentication, communicate mles and consequences, and initiate up- 
front interaction with die rules (selection of quality-of-service (QoS), billing, etc ) The 
user's communication witir CMPS 2302 is conducted through a non-standardized chamiel 
(e.g., Lme 23 16). The CMPS designer may provide an independent API for framing these 
interactions. 

Feedback Path 2313 from Composite and Render block 2309 serves an important 
purpose. The path is used to cross check that the system actually presented the user witii a 
given scene. Elementary streams that are processed by their respective modules may not 
necessarily be presented to the user. Furthermore, there are several fraud scenarios wherein 
an attacker could pay once and view multiple times. The feedback path here allows CMPS 
2302 to cross check the rendering and Uiereby perform a more accurate accounting. ITiis 
feedback is implemented by forcing the Composite and Render block 2309 to issue a start 
event tiiat signals the initiation of a given object's rendering that is complemented by a stop 
event upon termination. The feedback signaling process may be made optional by 
providing a CMP-notification flag that may be toggled to indicate whether or not CMPS 
2302 should be notified. AU CMPOs would be required to cany tiiis flag. 

The final modification to the structure is to require that the clear text buffers in the 
AVO, SDG and Object Descriptor processors and in the Composite-and-Render block be 
secured. This is to prevent a pirate from stealing content in these buffers. As a practical 
matter, this may be difficult, since tampering with these structures may well destroy 
synchronization of the streams. However, a higher state of security would come from 
placing these buffers into a protected processing environment. 

CMPS 2302 governs the fimctioning of Player 2301, consistent with the following: 
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• Communication mechanism between CMPS 2302 and the MPEG-4 player (via 
CMPOs) 

• A content release and decryption subsystem 

• Version authentication subsystem 

• Sufficient performance so as not to interfere with the stream processing in the 
MPEG-4 components 

CMPS 2302 may have a bi-directional side-channel that is external to the MPEG-4 
player that may also be used for the exchange of CMP information. Furthermore, the 
CMPS designer may choose to provide a user interface API that provides the user with the 
ability to communicate with the content and rights management side of the stream 
management (e.g., through Line 2316). 

Encrypted content is decrypted and released by CMPS 2302 as a function of the 
rules associated with the protected content and the results of user interaction with CMPS 
2302. Unenciypted content is passed through CMPS 2302 and is governed by associated 
rules and user interaction with CMPS 2302. As a consequence of these rules and user 
interaction, CMPS 2302 may need to transact with the SDG and AVO coding modules 
(e.g., 2310, 231 1) to change scene structure and/or the QoS grade. 

Ultimately, the CMPS designer may choose to have CMPS 2302 generate audit trail 
information that may be sent to a clearinghouse authority via CMPS Side Channel Port 
20 23 1 8 or as encrypted content that is packaged in the MPEG-4 bit stream. 

The MPEG-4 vl Systems CD uses the term "object" loosely. In this document, 
"object" is used to specifically mean a data structure that flows fiora one or more of the 
data paths in Figure 23. 

Using multiple SD-graph update streams, each with its own CMPO, allows an 
author to apply arbitrarily specific controls to the SD-graph. For example, each node in the 
SD-graph can be created or modified by a separate SD-graph update stream. Each of these 
streams will have a distinct CMPO and ID. Thus, the CMPS can release and decrypt the 
creation and modification of each node and receive feedback information for each node 
individually. The practical implications for controlling release and implementing 
consequences should be comparable to having a CMPO on each node of the SD-graph, 
without the costs of having a CMPO on each SD-graph node. 

Principles consistent with the present invention may be Ulustrated using the 
following examples: 

In the first example, there is a bilingual video with either an English or French 
soundtrack. The user can choose during playback to hear either the English or French. The 
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basic presentation costs $1. If the French soundtrack is presented there is a $0.50 surcharge 
If the user switches back and forth between French and English, during a single viewing of 
the presentation, the $0.50 surcharge will occur only once. 

In this example, there will be four elementary streams: 

The Scene Description Graph Update stream will have a CMPO. The CMPO will 
imply a $1.00 fee associated with the use of the content. TTie scene description graph 
displays the video. English audio and puts up a button that allows the user to switch to 
French. If the user clicks that button, the English stops, the French picks up from that point 
and the button changes to a switch-to-English button. (Optionally, there may be a little 
dialog at the begimiing to allow the user to select the initial language. TTus is all easy to do 
in the SD graph.) 

The Video Stream with the CMPO will say that it can only be released if the scene 
description graph update stream above is released. 

The English Audio Stream will be similar to the Video stream. 
The French Audio Stream will be similar to the Video stream but there is a $ 50 
charge it if is seen in the feedback chamiel. (The CMPS must to not count twice if the user 
switches between the two in a single play of the presentation.) 

An important requirement is that the ID for the SD-graph update stream appears in 
the feedback path (e.g.. Feedback Path 2313). This is so CMPS 2302 knows when the 
presentation stops and ends so that CMPS 2302 can correctly bill for the French audio. 

The rules governing the release of the video and audio streams may include some 
vanations. The rules for these streams, for example, may state something like "if you don't 
see the id for the scene description graph update stream X in the feedback chamiel, halt 
release of this stream." If the main presentation is not on the display, then the vidio should 
not be. This ties the video to this one presentation. Using the video in some other 
presentation would require access to the original video, not just this protected version of it. 

In a second example, an author wants to have a presentation with a free attract 
sequence or "trailer". If the user clicks the correct button the system moves into the for-fee 
presentation, which is organized as a set of "acts". 

Multiple SD-graph update streams may update a scene description graph. Multiple 
SD-graph update streams may be open in parallel. The time stamps on the ALUs in the 
streams are used to synchronize and coordinate. 

The trailer and each act are represented by a separate SD-graph update stream with a 
separate CMPO. There is likely an additional SD-gmph update stream that creates a simple 



10 



wo 99/48296 

PCT/US99/05734 

-74- 

root node that is invisible and sUent. This node brings in the other components of the 
presentation as needed. 

The foregoing description of implementations of the invention has been presented 
for purposes of Ulustmtion and description. It is not exhaustive and does not limit the 
mvention to the precise form disclosed. Modifications and variations are possible in light 
of the above teachings or may be acquired fiom practicing of the invention. For example 
the described implementation includes «)ftware but the present invention may be 
implemented as a combination of hardware and software or in hardware alone TTie 
mvention may be implemented with both object-oriented and non-object-oriented 
programming systems. The scope of the invention is defined by the claims and their 
equivalents. 
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We claim: 

1 . A streaming media player providing content protection and digital rights 
management, including: 

a port configured to receive a digital bit stream, the digital bit stream including: 
content which is enciypted at least in part, and 

a secure container including control ia&nnation for controlling use of the 
content, including at least one key suitable for decryption of at least a portion of the 
content; and 

a control arrangement including: 

means for opening secure containers and extracting cryptographic keys, and 
means for decrypting the encrypted portion of the content. 

2. The player of Claim 1 in which the digital bit stream includes at least two 
sub-streams which have been muxed together, at least one of the sub-streams including 
compressed information, and 

wherein the player further includes: 

a demux designed to separate and route the sub-streams; 

a decompression unit configured to decompress at least one of the sub-streams the 
decompression unit and the demux bemg comiected by a pathway for the transmission of 
information; and 

a rendering unit designed to process decompressed content information for 
rendering. 

3. The player ofClaim 2, fiirther including: 

a stream controUer operatively comiected to the decompression unit, the stream 
controUer including decryption functionality configured to decrypt at least a portion of a 
sub-stream and pass the decrypted sub-stream to the decompression unit. 

4. The player of Claim 3, further including: 

a patii between the control arrangement and the stream controller to enable tiie 
control arrangement to pass at least one key to the stream controUer for use with the stream 
controller's decryption functionality. 

5. The player ofClaim 4, further including: 

a feedback path fi:om the rendering unit to the control arrangement to allow the 
control arrangement to receive information fiom tiie rendering unit regarding the 
identification of objects which are to be rendered or have been rendered. 

6. The player ofClaim 1, wherein die digital bit stream is encoded in MPEG-4 

format. 
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7. The player of Claim 1 . wherein the digital bit stream is encoded m MPS 

format. 

8. Hie player of Claim 4, wherein the control arrangement contains a rule or 
rule set associated with governance of at least one sub-stream or object 

9. The player of Claim 8, wherein the rule or rule set is delivered from an 
external source. 

10. The player of Claim 9, wherein the rule or rule set is delivered as part of the 
digital bit stream. 

1 1. The player of Claim 8, wherein the rule or rule set specifies conditions 
under which the governed sub-slream or object may be decrypted. 

12. The player of Claim 8. wherein the rule or rule set governs at least one 
aspect of access to or use of the governed sub-stream or object. 

13. The player of Claim 12, wherein the govemed aspect includes making copies 
of the govemed sub-stream or object. 

14. The player of Claun 12, wherein the govemed aspect includes transmitting 
the'govemed sub-stream or object through a digital output port. 

15. The player of Claim 14, wherein the rule or mle set specifies that the 
govemed sub-stream or object can be transferred to a second device, but rendering of the 
govemed sub-stream or object must be disabled in the first device prior to or during the 
transfer. 

16. The player of Claim 15, wherein the second device includes rendering 
capability, lacks at least one feature present in the streaming media player, and is at least 
somewhat more portable than the streaming media player. 

17. The player of Claim 11, wherein the control arrangement contains at least 
two rules governing access to or use of the same govemed sub-stream or object 

1 8. The player of Claim 17, wherein a first of the two rules was supplied by a 
first entity, and the second of the two rules was supplied by a second entity. 

19. The player of Claim 18, wherein the first rule controls at least one aspect of 
operation of the second rule. 

20. The player of Claim 12, wherein the govemed aspect includes use of at least 
one budget 

21 . The player of Claim 12, wherein the govemed aspect includes a requirement 
that audit information be provided, 

22. The player of Claim 1 . wherein the control arrangement includes tamper 
resistance. 
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23 . A digital bit stream including: 

content information that is compressed and at least in part encrypted; and 
a secure container including 

governance information for the governance of at least one aspect of 
access to or use of at least a portion of the content information; and 

a key for decryption of at least a portion of die encrypted content 
information. 

24. The digital bit stream of Claim 23, wherein the content information is 
encoded in MPEG-4 format. 

25. The digital bit stream of Claim 23, wherein the content information is 
encoded in MP3 format. 

26. A method of rendering a protected digital bit stream including: 
receiving the protected digital bit stream, 

passing the protected digital bit stream to a media player, 

the media player reading first header information identifying a plugin used 
to process the protected digital bit stream, the first header information 
indicating that a first plugin is required; 

the media player calling the first plugin; 

the media player passing the protected digital bit stream to the first plugin; 
the first plugin decrypting at least a portion of the protected digital bit stream; 
the first plugin reading second header information identifying a second plugin 
necessary in order to render the decrypted digital bit stream; 
the first plugin calling the second plugin; 

the first plugin passing the decrypted digital bit stream to the second plugin; 
the second plugin processing the decrypted digital bit stream, the processing 
including decompressmg at least a portion of the decrypted digital bit stream; 
the second plugm passing the decrypted and processed digital bit stream to the 
media player, and 

the media player enabling rendering of the decrypted and processed digital bit 
stream, 

whereby die furst plugin may be used in an architecture not designed for 
multiple stages of plugin processing. 
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